2018 Marie Curie Individual Fellowship (H2020-MSCA-IF-2018)

AdinaBabesh
Posts: 85
Joined: Wed Jan 23, 2019 4:24 pm

Re: 2018 Marie Curie Individual Fellowship (H2020-MSCA-IF-2018)

Post by AdinaBabesh » Wed Feb 13, 2019 3:39 pm

In the evaluation letter, it is this:

"You may request an evaluation review on the procedural aspects of the evaluation (not the merits of your proposal). This request must be submitted by the coordinator (via the following link: https://webgate.ec.europa.eu/redress-fr ... work.iface) — within 30 days after receiving this letter."

AdinaBabesh
Posts: 85
Joined: Wed Jan 23, 2019 4:24 pm

Re: 2018 Marie Curie Individual Fellowship (H2020-MSCA-IF-2018)

Post by AdinaBabesh » Wed Feb 13, 2019 3:57 pm

Information on the means of redress
You may request an evaluation review on the procedural aspects of the evaluation (not the merits
of your proposal). This request must be submitted by the coordinator (via the following link:
https://webgate.ec.europa.eu/redress-fr ... work.iface) — within 30 days after receiving this letter.
You may request a legal review of the procedural aspects of the evaluation (not the merits of your proposal)
under Article 22 of Council Regulation No 58/20031
(‘Article 22 request’) — within 1 month of receiving
this letter (via the following link: EAC-FOR-APPEALS-UNDER-ART-22-OF-REG-58-2003@ec.europa.eu).
You may bring an action for annulment under Article 263 of the Treaty on the Functioning of the European
Union (‘Article 263 action’) against the Agency — within 2 months of receiving this letter.
Please be aware that you cannot take more than one formal action at a time. Thus, if you make, for instance,
a request for evaluation review, you cannot — at the same time — take any other action (e.g. also file an
Article 22 request or an Article 263 TFEU action). If you file an Article 22 request, you cannot — at the same
time — bring an Article 263 action.
You must wait for the final decision of the Commission/Agency and can then take further action against that
decision. All deadlines will start to run from when you receive the final decision.

AdinaBabesh wrote:
Wed Feb 13, 2019 3:39 pm
In the evaluation letter, it is this:

"You may request an evaluation review on the procedural aspects of the evaluation (not the merits of your proposal). This request must be submitted by the coordinator (via the following link: https://webgate.ec.europa.eu/redress-fr ... work.iface) — within 30 days after receiving this letter."

Geezer_MSCA
Posts: 27
Joined: Sat Feb 09, 2019 3:48 pm

Re: 2018 Marie Curie Individual Fellowship (H2020-MSCA-IF-2018)

Post by Geezer_MSCA » Wed Feb 13, 2019 4:07 pm

IF ST LIF wrote:
Wed Feb 13, 2019 3:26 pm
p3dr0 wrote:
Wed Feb 13, 2019 12:56 pm
CountZ wrote:
Wed Feb 13, 2019 10:49 am
Hi again everyone

We are writing a complaint to the Commission about unfair evaluation processes, but we need your help.
If you think that the evaluation is unfair, please tell us why. Maybe you got a much lower score on re-submission. Maybe you don't think your proposal should have received the high/low score it did. The goal is the get the Commission to rethink its evaluation process, so as to make this less of a lottery, and instead to reward the best proposals.
My two cents on this.

You cannot fight against unfair (or perceived as unfair, which is a very important nuance) reviews with limited review time and multiple reviewers. If you believe it was unfair, as stated by the EC "you may request an ‘evaluation review’ on the procedural aspects of the evaluation (not the merits of your proposal)". I did it for what I considered being an unfair review two years ago, it did not change anything. Unfair, yes, but that's the name of the game.

What is quite different, in my opinion, is getting a much lower score on re-submission. Here it shows that basically the full evaluation process is flawed, and that the full thing is a lottery (actually, some authors have suggested using a partial lottery system, see eg Gross & Bergstrom 2019 PLOS Biology).

If you want to follow the op-ed route, the second phenomenon appears more problematic to me (differences of more than 20% in the final mark from the same starting material???). The first one is trivial: of course you may get conflicting reviews depending on the level of expertise of the reviewers within the panel. Still, if you want to discuss this "unfairness", is the number of evaluation reviews asked per year available? This could be a good indicator. Another thing to keep in mind: for one unfair review, how many fair ones? And which ratio is acceptable?
Totally agree. This is crystal clear indicator of the weaknesses of the system. I've been discussing this morning with the OIP office of my host and they've told me that almost every year they have at least one case that gets a big drop after a resubmission. This is a real issue with their system, lack of consistency. It's a total flaw that the strengths of the first year turns into weaknesses based on the subjective view of an "expert" -I would like to see the name of the evaluators- that spend few minutes per application.
For you and all those who want to have an idea of who the experts are, search on Google the following

h2020-expertslists-excellent-msca-2016_en

The first result should be an excel file (of the same name) with the list of all the evaluators for the 2016 MSCA-IF

p3dr0
Posts: 3
Joined: Tue Feb 12, 2019 10:49 am

Re: 2018 Marie Curie Individual Fellowship (H2020-MSCA-IF-2018)

Post by p3dr0 » Wed Feb 13, 2019 4:37 pm

CountZ wrote:
Wed Feb 13, 2019 2:49 pm
Thanks for this!

We were considering the article on the lottery system. I still have to read it. My opinion (without having read the article yet) is that such a system would fail to differentiate between very good proposals (that should get funded) and good proposals (that shouldn't be funded given limited resources).
p3dr0 wrote:
Wed Feb 13, 2019 12:56 pm
Still, if you want to discuss this "unfairness", is the number of evaluation reviews asked per year available? This could be a good indicator.
What do you mean by the number of evaluation reviews per year?
You're welcome!

For the lottery system, at some point mark differences start to be subjective: is a proposal ranked 95.6 really much better than a proposal ranked 95.3? Instead, we can first identify all very good proposals (say, above 90) and then draw randomly from this pool of proposals. One just has to reach some level of "excellence" (I hate this word) to access the lottery, but then s/he could apply every year with her/his very good proposal until s/he get funded, without spending too much time writing/editing the damn thing again and again. In the current system, one can spend as much time running behind funding than doing actual research.

For the evaluation review, sorry, it was a bit cryptic, but as answered above, these are asked by the applicant when s/he feels the review was unfair. I might assume the number of evaluation reviews per call could be a good indicator of the perceived fairness of the evaluation, say if there are a lot of such evaluation reviews for every call this could indicate a problem. I am not sure the EC publishes stats on these, though.

hopefulacademic
Posts: 24
Joined: Sat Feb 09, 2019 2:09 pm

Re: 2018 Marie Curie Individual Fellowship (H2020-MSCA-IF-2018)

Post by hopefulacademic » Wed Feb 13, 2019 4:46 pm

We have to take into account the differences in evaluation by different reviewers from different countries. I remember from my time studying in Belgium that grading was much more negativist: deducting points, rather than looking for the positives. People who came from abroad were surprised that even with a near perfect task done, they still got less-than-perfect scores. This probably has happened here as well but as long as you work with humans, this is possible. This is not a call for algorithmic evaluation though, as that would really allow for gaming the system.

Simbi
Posts: 13
Joined: Fri Feb 08, 2019 10:02 am

Re: 2018 Marie Curie Individual Fellowship (H2020-MSCA-IF-2018)

Post by Simbi » Wed Feb 13, 2019 6:51 pm

I am a clear example of the randomness/weakness of the system. I first submitted last year and I got 91, this year I went down to 76...same identical proposal. I am literally speachless.

CountZ
Site Admin
Posts: 154
Joined: Thu Jun 01, 2017 7:14 pm

Re: 2018 Marie Curie Individual Fellowship (H2020-MSCA-IF-2018)

Post by CountZ » Wed Feb 13, 2019 6:52 pm

Simbi wrote:
Wed Feb 13, 2019 6:51 pm
I am a clear example of the randomness/weakness of the system. I first submitted last year and I got 91, this year I went down to 76...same identical proposal. I am literally speachless.
Wow what a huge jump!
Did you address the weaknesses identified in the proposal that scored 91?
Was it marked as a resubmission?

Simbi
Posts: 13
Joined: Fri Feb 08, 2019 10:02 am

Re: 2018 Marie Curie Individual Fellowship (H2020-MSCA-IF-2018)

Post by Simbi » Wed Feb 13, 2019 6:54 pm

Yes and yes. I have carefully addressed the weakness.
CountZ wrote:
Wed Feb 13, 2019 6:52 pm
Simbi wrote:
Wed Feb 13, 2019 6:51 pm
I am a clear example of the randomness/weakness of the system. I first submitted last year and I got 91, this year I went down to 76...same identical proposal. I am literally speachless.
Wow what a huge jump!
Did you address the weaknesses identified in the proposal that scored 91?
Was it marked as a resubmission?

CAR-2018
Posts: 9
Joined: Wed Jan 02, 2019 5:01 pm

Re: 2018 Marie Curie Individual Fellowship (H2020-MSCA-IF-2018)

Post by CAR-2018 » Wed Feb 13, 2019 9:51 pm

CountZ wrote:
Wed Feb 13, 2019 11:17 am
btw these people have been added to the group
Geezer_MSCA
Thana2019
CAR-2018
Sorry, what's the link to the group? Thanks.

Bren
Posts: 349
Joined: Sat Jan 12, 2019 11:55 pm

Re: 2018 Marie Curie Individual Fellowship (H2020-MSCA-IF-2018)

Post by Bren » Wed Feb 13, 2019 9:52 pm

Kitten?
Last edited by Bren on Wed Feb 13, 2019 11:49 pm, edited 1 time in total.

Post Reply