Since there are no currently active contests, we have switched Climate CoLab to read-only mode.
Learn more at https://climatecolab.org/page/readonly.
Skip navigation

Community Discussions

The judges written 'Evaluation' of proposals are not showing up on many proposals; Also are there standards for judging?

8comments
Share conversation: Share via:

Michael Hayes

Aug 4, 2015
05:12

Member


1 |
Share via:
It would be nice to have at length feedback from the judges posted on the page provided (a.k.a the Evaluation tab/page). That feedback is important on many levels to the proposal developer side of the table yet is minimal at best and often simply not provided. Most of the minor comments I have received are extremely weak on rationale, largely off the main STEM topic and or simply non-extant. In simple words, they seem to be pulling the reasoning/rationale out of their hats...if they even bother to give a reason. I'm trying to develop a better feel for how best to write for what has been truly unpredictable, if not simply arbitrary, judging standards and practices. The same proposal can get completely different responses from every panel judging...the same proposal. Many times the same proposal can simply be rejected for the most ill-logical and simply unsupportable reasons or simply dropped without meaningful comment(s). Thus, there seems to be little, if any, standards being applied to the judging and (hopefully) by accumulating a large number of comments from the judges (on one specific proposal), it may be possible to better judge/predict future...well...emotional moods....of the judges (no detectable level of collective logic standards on the judging so far). Michael

Annalyn Bachmann

Aug 6, 2015
11:32

Staff


2 |
Share via:
Hello Michael, Thank you for reaching out to us! I am very sorry that you are displeased with the quality of the comments you received from our Judges. There are actually judging standards that our Judges do abide by, which can be found in the Resources section of the contest. However, you can always reach out to the Fellows of the contest to see if they can add any additional comments! Please let me know if I can do anything else to help make this process easier for you! Thank you, Annalyn

Gabriel Harp

Oct 6, 2015
12:12

Member


3 |
Share via:
Hi Annalyn, It's not actually about making the process easier; it's about building engagement. My own experience, like that of the above comment, has been extremely disappointing. At the end of the day, Climate CoLab fails to build community because there isn't earnest engagement from judges or others. But what makes it worse is that the platform team seems to disregard comments to that effect. I guess I'm glad that the effort can continue without authentic responses from the team running the contest, but I suspect that it will be short lived if meaningful changes aren't made. kind regards, GH

Michael Hayes

Oct 9, 2015
04:21

Member


4 |
Share via:
Gabriel, Did you, yourself, re-post twice? If you did not re-post, please contact me at voglerlake@gmail.com I would like to start an association of authors on the rationale that we, as human research subjects, are entitled to certain rights and, in some instances, those rights...may(?)...have been breached by those entrusted to protect the human subject's rights within this experiment. Best regards, Michael

Michael Hayes

Oct 9, 2015
05:34

Member


5 |
Share via:
As a post script: Regardless of the judge's opinion of a proposal, language such as "This proposal is a muddle of technical and financing options" conflicts with the human research subject's right to be dealt with in a respectful and dignified manner. As this experiment is highly public and can have a wide ranging ripple effect on the research subject and even the concept(s) that they champion, the conduct standards of the authority figures in this experiment should not simply scrap by at the bottom of acceptable professional ethical standards but should be exemplary to the point of setting new standards in the field. Regrettably, if a single research subject complains about what they believe is sub-standard ethics/practices, they will most likely be classified in a negative way and be subject to number of 'computer glitches' to discourage their further involvement in the experiment. One common discouraging 'computer glitch' is to send the complaining research subject not one proposal rejection notice but...60...notices on the same proposal rejection. This has happened to more than one research subject on more than one occasion. The list of these harassment 'computer glitches' is rather long and has been reported to me by multiple human research subjects. As such, by initiating the CoLab Human Research Subjects Association (CHRiSA), we can protect ourselves from someone working in the CoLab from, figuratively speaking, simply walking by and rattling the cage of an un-favored research subject (clearly an act of sociopathy) as well as protecting ourselves from being treated by judges in less than a dignified manner. I would like to hear back from authors who believe they...may...have been subjected to sub-standard practices and or ethics. Together, we can be a strongly positive influence on the future of this experiment as well as protect our inherent rights as human research subjects. Best regards, Michael

Annalyn Bachmann

Oct 25, 2015
01:51

Staff


6 |
Share via:
Hello Michael and GH, I am very sorry that you are so disappointed in the quality of the judging process. We're always working to improve the Climate CoLab, and we're definitely taking your comments into consideration. They will not be ignored. We appreciate all of our community members and value your comments. We want to assure you that we will continue to improve the contest process, and hope to incorporate our community's comments about the platform in the next round of contests. Thank you for your understanding and we hope that we can work harder to improve the Climate CoLab to your satisfaction. Thank you, Annalyn Bachmann Project Assistant with the Climate CoLab

Michael Hayes

Oct 28, 2015
09:37

Member


7 |
Share via:
Dear Annalyn, The judging process has a great deal of good to be said about it. In my dealings with the judges, I have found truly remarkable examples of excellenct (i.e. Karim-Aly Kassam et al.) judges. Obviously, the likes of a Dr. Kassam would not post an unprofessional comment such as "This proposal, if you can call it that.". Yet, such comments are found in the record. Also, is there any reason why comments are being repeatedly posted? Best regards, Michael

Annalyn Bachmann

Nov 10, 2015
05:25

Staff


8 |
Share via:
Hi Michael, I am not particularly sure why that is happening with the comments, but I'll report it to the development team as soon as possible. I will also be sure to pass along your comments about judging as everything helps us improve the Climate CoLab. Thanks, Annalyn