2008-07-08: [IFESS 2008] TPC reviews

Andy Hoffer wrote 08.07.2008 22:12
===========================

Dear Thomas, Martin, and members of the IFESS 2008 Scientific Committee,

Having reviewed a number of the submitted papers, I find it disappointing that several contributions really don't meet a minimum quality level to recommend acceptance. It is not just a matter of unreadable English, or disregard for presentation structure guidelines -- some submissions are so scientifically deficient that, if they were research papers submitted by undergraduate students for a course, they would not deserve a passing grade.

I wonder what the Program Committee plans to do about scientifically marginal contributions. On one hand, one would like for our organization to be inclusive and encourage participation. As well, exposure to higher quality science presented at the conference by good speakers can be formative especially for inexperienced researchers. On the other hand, we must control the quality of our published contributions and ensure continued respect for IFESS and our disciplines. This is an important dilemma. We should have in place clear requirements for paper acceptance. I would appreciate hearing your opinions.

Best regards,


Andy Hoffer

Comments

Janis Daly: 9.7.2008, 17:16
===========================

Hello Andy and Everyone,

It is important that you are bringing up this situation. If there are no agreed abstract review standards of acceptance, then if we care about the quality of the posters and papers, we should develop them and disseminate the acceptance standards at the time of call for abstracts, at least for next year. It would seem important for us all to put our minds to this before our Board meeting so that we can have an agreeable draft of standards to consider and hopefully approve at the next Board meeting.

That said, it does not solve the situation for the coming fall conference.

1. One way to handle it this year is simply to accept everything, since no mention was made about standards or criteria for acceptance.
2. Another way to handle it this year would be to simply reject the submission. This might seem heartless, especially since acceptance standards don't seem to exist and were not known, up front, to the submitters.
3. Another way to handle it just for this year would be to identify a "mentor" for each person who submitted a paper that is not up to acceptable standards. The mentor would work with the submitter until the abstract is in acceptable form. The question would still remain as to how to handle a subsequent situation if the paper is simply not salvagable (maybe the work was not yet completed and can't be done before the acceptance decisions have to be made).

Just some thoughts to consider. There may be other options that would be preferable.

Best regards,
Janis Daly

Manfred Bijak wrote
====================

Hello dear BoD members!

Sad that we have to raise a discussion about the quality of the papers for the IFESS conference.
Thank you Janis for the input, I guess we all agree with you. Quality of submissions will be definitely a topic during our meeting in Freiburg.

To be realistic, we have to look at this problem from two sides, from IFESS side where we have to keep the paper standards acceptable high and from the side of the organizer who is completely dependent on the number of participants to make the conference a financial success - or at least to avoid a financial disaster.

Anyway, for the next conferences we have to have published standards for submitters. But, as conference organizer myself I know that one can publish whatever, only few people take care (or even read it).

So it seems the only solution which covers the needs of IFESS and organizers to establish a guiding review process like Mohammad Sawan did it during the conference in Montreal. He earned lots of honors for this review process, especially from students. They welcome it to have a real helping hand when writing (a first) extended abstract.
On the other side Mohammad stated that this process was extremely exhausting and time consuming.

For this year it is a question of time when Thomas has to have the camera ready manuscripts in his hands.

If possible - Thomas would you ask reviewers to contact the authors of the worst papers and to send them an "invitation to improve submission" within a for you acceptable time?
If the papers are still not appropriate they should be rejected.

Any other ideas or possibilities?

Best regards

Manfred

Glen Davis: 9.7.2008
====================

Dear Andy and colleagues

This is just a thought, but I will relate how the problem of uneven Abstract quality was improved (not resolved) in my "other" professional organisation (American College Sports Medicine).

Faced with a similar dilemma of uneven Abstract quality and with a high number of submissions (ACSM is a BIG meeting), the College instituted a policy whereby Abstracts would not be refereed IF they were signed off by a Fellow of the College. I.E. If you were a postgraduate student or an academic of NESB, then you could get guaranteed acceptance if a Fellow took responsibility. Now, we don't have Fellows (but we could), and most Fellows of Societies/Colleges are well-published and senior academic/clinicians.

But once the College did that, the quality of abstracts improved, since Fellows would not be keen to "sign off" if the writing was poor or the research sub-par (and the Fellows pointedly alerted the abstract author to that).

Having been a Chair of a previous IFESS Annual Meeting, I would not be so enthusiastic to place the burden of abstract mentoring upon the Organising Committee or the Technical Committee (although we did just that in 2003 for the Australia meeting) - it is a big job!

That's my $0.02

Cheers

glen

July 9, 2008 9:56 AM

Ross Davis: 09.07.2008 22:18
============================

Dear BoD,
Just to give some IFESS history: Because each of the 3 world areas that we meet in rotation, the BoDs have understood that this is a one in a 3-yearchance for Students and some Faculty in that world area to present for maybe their first opportunity.
As a results their proposed Papers are not always at the usual level that we expect; however, we must do all we can to give the inexperienced presenter guidence to develop their skills and continue to grow with the Society. It would be important to known fron Andy H. how many are first-time presenters and need caring help to develop their skills and grow with the Society.
I remember when we first started IFESS, some BoDs discussed meeting every 3 years instead of annually. The strong response from the majority was to continue with yearly meetings, in order to bring in new members in each of the 3-world areas, so adding to our strength and diversity.
Best regards,
Ross

July 10, 2008 1:13 AM

Dejan Popovic: 09.07.2008 22:35
================================
Dear all,
I stongly support that we review submissions, and accept only papers that meet
basic criteria: some novelty clearly expressed, technically sound, and with
adequate references.
I do not see who is benefitting if papers that do not meet basic standards are
accepted for confernece proceedings? Some of the papers from previous
Conference proceedings are cited and used as milestones in research, and we
should try to encourage this even more.
Regards,
Dejan Popovic

July 10, 2008 1:15 AM

Milos Popovic: 10.7.2008 00:13
==============================
Hi,

I share Dejan's opinion.

The papers have to be of appropriate level to be accepted for presentation
at the conference. I also believe that the review process that Mohammed did
in 2005 was excellent, and that providing feedback how to further improve
papers was useful to all participants.

We should try to stick to processes that have been proven in the past. The
review process and rejection of poor papers has been found to be a good
process, and we should stick to it. If Thomas needs help to process the
papers please let me know. My people and I are ready to help. Also,
processing papers using web based solutions (which are not cheep but are
fast and elegant) is a good approach. I've done it 3-4 times already with
the conferences I have organized, it works like a charm.

Best wishes,
Milos

Hi,



The approach we had for the national SCI conference in last 6 years was the following:



* All papers are reviewed by at least 2 external reviewers according to set standards (I can share them with you)
* If paper meets the minimum standards it is accepted and comments how to improve the paper are given to the authors, regardless if the authors are junior or not. And they were given time to resubmit an improved paper.
* If the paper was substandard it was rejected
* Papers advertizing products were automatically rejected and these people were advised to purchase a presenter’s space at the conference
* Top 12 papers received awards (3 in research, 3 in education, 3 in patient care and 3 student papers)



This process improved submissions in year 1 considerably, in years 2 and 3 the level of papers increased by itself. This worked well for us. Right now we receive 80-100 papers per conference and there is very little we need to do about their quality, actually student papers are often better quality than “fellow papers”.



I hope this could be of help.



Best wishes,

Milos

-----------------

July 10, 2008 1:17 AM

Thomas Stieglitz: 10.07.2008 17:37
=================================

Dear all,

let me circulate some of my thoughts with respect to all of your emails.

I really appreciate your comments and we definitely should have a time slot in the BoD meeting to discuss about an opportunity to increase the quality of submissions.

When doing the assignments and some of the reviews I got the impression that our problem is definitively NOT that students do not know how to write and to lend them a hand and to teach them. I never found a paper with a single name on it (at least not from a junior person). According to good scientific practice honorary authourshi is not allowed and senior scientists should at least read the papers on which their names occur.
I guess the problem is the attitude of (hopefully only some) persons to the IFESS meeting. We already had some comments like "I only like to submit an abstract" (where I cannot be nailed) to IFESS and publish later on in a "good" journal. Nobody would try to submit the same paper twice to an IEEE conference. We got one that was identical to an IEEE conference proceedings paper (and that from a senior person). As long as people think everything that can be written on three pages within one hour is good enough for IFESS, we definitively have a problem. The most lousy papers that I have reviewed have been this year with at least one senior author on the author list.

Martin and I will go through the reviews and will reject the papers that represent plagiarism and that are really lousy.
Next year, the standards should be communicated at the earliest possible point of time and authors should definitely sign something to be aware of the regulations.

That's all for the moment. I have to go back to the IFESS papers.....

Best wishes from sunny Freiburg

Thomas

July 11, 2008 1:38 AM

randomness