QIP is in Beijing this time from Jan 21-25, 2013. And the call for papers has just been posted.
- Submission deadline for talks: October 5, 2012
- Notification for talk submissions: November 9, 2012
- Submission deadline for posters: November 16, 2012
- Notification for poster submissions: November 26, 2012
The submission guidelines continue to slowly evolve. Last year called for “2-3 pages in a reasonable font” and this year is a somewhat more precise “up to 3 pages in 11pt font with reasonable margins, not including references”. From my experience on PCs in the past (I’m not on the PC this year, though), I’ll mention that these short submissions sometimes are very good, but sometimes aren’t addressed at the key questions that the PC will want to consider, such as:
- What is the key result?
- How does it compare with previous work on this subject? And how does it fit into the broader context of other research?
- What are the technical innovations necessary to achieve it, and possibly to overcome obstacles that stopped people before?
- (If the topic area is unconventional, or the submission is about a newly invented problem.) Why is this a good area for people to study?
These are kind of basic, but it’s surprising how many good results are undermined by not explaining these points well.
Another feature of the CFP to notice is that authors are strongly recommended to link to a version of their paper on their arxiv, or to include a long version of their paper as an attachment. The idea here is that if a result is not ready to go on the arxiv, then it’s less likely to be correct. There was debate about making it mandatory to post a full version on the arxiv, but this idea was dropped because many people felt that this way we might miss out on the freshest results.
A related concern is about keeping the barriers to submitting to QIP low. Of course, travel is always a barrier, but hopefully we shouldn’t waste a lot of people’s time with elaborate submission requirements if we expect the acceptance rate to be around 20%. I think the current requirements are reasonable, as long as people don’t stress too much about making a special short version, e.g. by writing something informal about why the paper should be accepted and referring liberally to the long version instead of trying to pack as much as possible in three pages. But I can imagine other systems as well. What do you think about the way QIP is being run right now?
My two cents: I like the idea of only accepting papers that are already on the arXiv. If a result isn’t ready in time, then we’ll get to hear about it next year. I don’t think we have to worry that much about freshness… only bad results get stale.
Please let me agree whole-heartedly with Steve Flammia’s post (above). And it’s terrific when slides and/or videos of talks are posted on-line — the more timely the posting, the more effective becomes the communication of enjoyment and adventure in research.
Let me be a curmudgeon (not a far step, I think.)
Page limits? Why? The answer you get is: to save the PC time. The answer I give: if your paper doesn’t have an introduction that doesn’t convince the committee, you’re doing it wrong (I did it wrong most of my life.) Details can only add, not detract, from a paper? Page limits feel good for people who like rules.
Why does QIP continue to use a half-assed approach somewhere between physics conferences and computer science conferences? As if it isn’t hard enough to get a job in the field, why does one of the best conferences in the field insist on a procedure that handicaps job candidates?
Which one is more helpful for the program committee, whose 17 members collectively know a thing or two about quantum information and computation, but do not necessarily possess expertise in every possible subtopic of the field, to attempt to select the best possible talks for QIP: a 3 page, non-technical extended abstract providing an intuitive description of results, or a 45 page technical paper with all of the low-level details needed to establish the results? If you are in the minority that prefers the full technical paper for this particular purpose, then you should be happy to know that submissions must point to such a paper or include it as an attachment — but people should not expect the PC to read it if the extended abstract fails to make a good case for the talk. If a person is not capable of explaining their research results in a three page non-technical abstract, or can’t be bothered with doing this, how good is their 20-30 minute talk for a general audience of quantum information and computation theorists going to be?
For those (unlike Dave) who have not served on the QIP program committee, or another program committee like it, it is worth keeping in mind that each program committee member will be responsible for reading, understanding, and ranking somewhere around 25 or 30 submissions in less than one month (with whatever subreviewing help they can obtain, of course). The extended abstracts give us a fighting chance to do this properly, as opposed to having randomly chosen and often highly biased experts assigning nonuniform scores to papers and calculating the averages.
Regarding page limits, font sizes, and so on, I fully admit that I like rules. But that is not the main reason for wanting to impose standards on submissions, nor is saving the PC time. (A much more effective way for any PC member to save time would have been to decline the invitation to serve on the PC, which of course is completely voluntary, mostly thankless, and a huge amount of work when taken seriously.) The real reason is fairness, to create as level a playing field as possible where people submitting work know what the PC is expecting and wishes to see. If, in addition, PC members don’t want to get eye-strain and headaches trying to read abstracts written in 9 point font with 5mm margins, that’s really not such a terrible crime. The existence of submissions like that, or that try to bend the rules as much as possible in other ways, is pretty much guaranteed if the call for submissions does not say otherwise.
Some might not think the process is actually fair; there may be misjudgments, and many will disagree with the outcomes, but in general I think the process works pretty well. I would personally prefer to think of the approach as being fully assed, or nearly so, taking the better aspects of physics and computer science conferences… but everyone is entitled to their own opinion. If anyone has specific suggestions for improving the process, the program committee and steering committee would certainly like to hear them. Those suggestions might not be followed, but I can guarantee you that they will be considered if they are both sensible and sincere.
Regarding job candidates, I’m not seeing the connection… the program committee’s job is to put together the best possible program for QIP that it can, not to get people jobs. If there really is something that the workshop is doing that is handicapping job applicants, however, that is something that should be discussed further: please explain the connection.
(Damnit typed a long reply on my iPad and the battery died. Why do you hate me Apple? WhyWhyWhy? Need to get an Android tablet apparently.)
In rebuttal to John, who is most certainly correct (it goes without saying, but this is the internet where opinions NEED TO BE EXPRESSED. Loudly. IN CAPS. All opinions below are mine, are mostly only designed to stir debate, and should not be used against me in a court of law.)
An animal that is half lion and half peacock is impressive if it gets the ferocious roar of the lion and the beautiful tail of the peacock, but is something quite less useful if it gets the head and the tail get set in a transposed state. QIP, unfortunately, does just this, mostly from first of all from the perspective of usefulness to people applying for jobs, but second of all from the perspective of actually support the kind of long term research that academia should be about.
I’ll start with the jobs, because that one I am more certain about (which means I’m definitely wrong, but hell, I’ve already flamed out of this game already.) John, do you really want to argue that conferences have nothing to do with getting a job in a CS department?!?! In an ideal world this would be true (“Marge, I agree with you – in theory. In theory, communism works. In theory.”) But in the world I witnessed, CS is a mixing pot of tons of different disciplines — judging across this divide is very difficult — and conferences provide a way to signal across this divide. From this perspective conferences are very important for job seekers and to deny this is to throw the junior people under the bus. It is to say that you will not help them, because they should be idealistic even while they are getting run over by a system that does work according to a flawed system. It is to have a community that does not want to help its members, because ideals matter more than helping drowning postdocs. It is a world I would want to live in, but alas it does not appear to be the branch of the universe my wave-function occupies.
Now as to QIP’s roll in this, I can only give a perspective on how much it matters to people who are applying to at least one top ten CS department in the US. It does not count. In fact it counts negative. I’ve actually heard (multiple) faculty say (paraphrasing) “well I wouldn’t support that person because they do quantum computing.” (I was lucky enough to be a part of a department that was very open about its hiring and that involved all the faculty, at some level, in the hiring process.) Want to have a shake of a chance at getting a job in a CS department if you do the kind of work presented at QIP? Win some FOCS or STOCS best paper/best student paper awards. QIP won’t help you, sorry. Why is this? Well what is a QIP paper? There isn’t one. It’s a summary, there are no proceedings. It’s not even a half-assed journal like the rest of computer science. How could that count for anything? As for whether it helps in physics. Well that’s the lion. Physicists don’t count conferences. So all conferences that you spend your time working on getting accepted to from a physics perspective basically end up like the posterior of a lion.
Now, you (rightly!) say, but conferences aren’t about getting hired! (It’s okay to be young and naive, in fact its a blessing, but to be young and naive and not secretly aware of the odds against getting an actual job in academia, well… saying conference aren’t tied to jobs is like saying I’m sane. Un. Likely.) Indeed, I don’t believe conferences should be about getting hired (I’m just pointing out I think the current model does not help at all, and actually hurts, and that pronouncements by people who have succeeded in the system are…suspect at best. Survivor bias is a bitch. Talk to me. I’ve failure bias!) So what are they about?
Are they about “giving a good talk?” as you suggest. Well Stephen Colbert would certainly give a good talk at QIP, but no one would like to see him at QIP would they? Shit, okay I’d want to see him give a talk. Someone should ask him to keynote. But more to the point, does having a three page summary really show that someone can give a good talk? I doubt it. I’d give my righty arm to hear a talk about any new result by Kitaev, but can I image him winning the 3 page beauty contest? (Why the right arm? I’m a liberal.)
The 3 page limit also has the very preverse effect of leading to submissions for work that fits neatly in exactly 3 pages. Or as I like to call it 3/4 of a PRL. A PRL is the smallest unit of physics publication, having the information content of exactly one bit. You can guess what that bit stands for, just let me say, it’s not a very useful bit. It completely discourages results that don’t fit nicely into a 3 page “synopsis”. I’d love to see a 3 pages summary of http://www.cs.bu.edu/faculty/gacs/papers/long-ca-ms.pdf
(the 40 page summary of that paper is pretty good http://www.math.umn.edu/~gray/postscript/gacs.ps so it would need a “summary of a summary” to get down to 3 pages?) I don’t want a summary of QIP=PSPACE. I want the full damn thing, screw dressing it up with 3 page lipstick.
[On a side note I’d also like to be convinced that conference paper are actually good indicators of future usefulness of a paper (how to do this when there is little non-anecdotal evidence about paper that were rejected seems hard. Simon’s algorithm anyone? Rejected papers that end up being important, how do we spot these?)]
But back to the more important subject, if conference aren’t just about giving a good talk, what should they be about? About producing the best set of speakers to best convey the most interesting and hopefully future important subjects. That sounds pretty close. So why are we limiting the pages? Because program committees couldn’t handle bigger loads? They can’t read an introduction and judge a paper from the introduction, just as they would a 3 page paper, but with the added benefit that they could ALSO read the full gritty details (and perhaps find flaws in those details which make them reject the paper?)
Which then brings me to the question of why we have program committees in the first place. Why are a chosen few to decide was is and was is not a good result? In this age of electronic tools, why don’t we democratize the process? Maximize what the conference goers themselves want to see, and crowdsource the problem. Why is the signal produced by a crazy website like, http://www.scirate.com/ not a good signal for who to invite to give talks at QIP?
Why not let the Plebes vote and open the door to Tribunes. The Patricians of the program committee win every time, game theory 101, by setting the agenda. QIP a few years back basically rejected every physics related talk that was submitted. Given the content of the program committee that year, that wasn’t too surprising (I’m not faulting them for doing this, I think it was an awesome group of speakers and program committee members, but just pointing out that the program committee has a very strong influence on the content of the conference.) Why don’t we invite all of the papers submitted to be reviewed by all of the people who are to attend? Distribute the workload, involve the community, trust in the basic decent honesty of the vast majority of people (make their votes public, exposing their biases to the world, instead of hiding it behind a non-transparent select group of the chosen few.)
Now as to QIP ever doing something like this. Ha! Sure the steering committee would listen, politely. But would they really change to a model that allows for long submissions? Not while they have to read them. Would they move to a system that is more transparent and eliminates the program committee itself? Why would the Patricians give rights to the Plebes. Especially when the Patrician’s themselves are all sitting nicely at the top of a big pile of tenured loot. There is no way they would consider such a radical change. So I’d hope they at least consider moving to a more traditional CS conference patern, with 10 page papers, etc. At least that way they’d be more likely to produce peacocks (and the Lions would migrate elsewhere, across manifolds smooth, to a land where they aren’t derided for studying two-level systems.)
I agree that there is a problem of CS depts not hiring quantum people, but I don’t know if this is because QIP and other quantum conferences aren’t clones of STOC/FOCS/etc. Many people in CS theory think that their conference system represents a bad equilibrium, which they wish they could escape.
In general, there are many departments that should be hiring a lot of people that they’re not. For example, “applied math” should be much bigger than just PDEs (e.g. including convex optimization), yet it’s often dominated by PDEs. “Classics” departments shouldn’t be just Greek + Roman studies, but should look at ancient civilizations in other parts of the world too. I could go on for awhile…
“Many people in CS theory think that their conference system represents a bad equilibrium, which they wish they could escape.” Indeed, and how much progress have they made sense we heard these arguments when we started as grad students, Aram? I find it more likely that changes to the system come from completely new entities, not old ones modifying themselves. I used to think quantum computing had a chance to do this, but has it?
I do not know the situation in CS, but most physicists do not consider contributed conference talks to have much prestige. I’m not saying that is the right thing for them to think, but that is how it is. Partly, this is because for many conferences, like the large APS meetings, basically every contributed talk is accepted. In fact, by far the majority of physicists do not know what STOC/FOCS are, much less realize that there is prestige associated with them. So, I am not sure what QIP can do to alleviate this. The recent change to calling the invited talks “plenary talks” helps, as that term has some more prestige. Perhaps best paper prizes would help; perhaps even a model like the Cannes film festival with both juried and audience prizes. Suggestions would be nice, but unless we actually go to having the contributed talks published in an established journal, just having 10 page papers for the talks in a published proceedings won’t mean much to most physicists. If, for example, QIC devoted 1 issue per year to QIP talks, then it might help as people could cite the journal reference, but equally people could just publish the paper in QIC so I am not sure it would be worth much.
Also, I think in CS the situation is that STOC/FOCS have prestige because they have prestige, not because they have proceedings. It’s like celebrities that are famous for being famous. Increasing the prestige of QIP would be a good thing for quantum computing and for those in the field, but how do we do this? I’m really curious about ideas on this.
Final comment: I think one main use of the 3 page restriction, etc… is that, well, some people submit papers missing section (like, literally with notes to coauthors like “Section 3.7, Proof of Main Theorem: Hey Bob, can you finish this part before we submit it? Section 3.8, …”), with parts of the paper completely out of order, etc… (to be fair, I’ve seen such submissions to STOC/FOCS too). So, give a good 3 pager, and also point to the 40 page arxiv, and that will be readable, hopefully.
Well, QC does a number of things better, maybe because it’s a new field. I think that QIP is a better model than STOC/FOCS, in part because there isn’t so much prestige attached to the talks. Also, the use of arxiv.org is good, although TCS is also starting to do that.
I very much doubt changing QIP to a standard FOCS/STOC-style conference with proceedings would help quantum information people get jobs, even in CS departments, and it might well hurt. In the current system, you can submit your paper to both QIP and FOCS/STOC and get accepted to both. If QIP had a proceedings, you would have to choose whether to advertise your result to quantum information people or to advertise it to CS people. I don’t see how that would help your job prospects.
The main barriers to getting QI people faculty jobs cannot be addressed by QIP, no matter what it does to its format. Someone who says “well I wouldn’t support that person because they do quantum computing” is not likely to suddenly change their mind based on QIP getting a proceedings.
When I write recommendation letters, I point out that getting a talk accepted at QIP is a big deal, and I’ve seen other people start to do this as well. That’s one way to get the word out, but it still only has meaning if the department is willing to consider hiring a QI person.
Crap why does this nesting of replies only go so deep. This in reply to Daniel Gottesman in his reply below:
“I very much doubt changing QIP to a standard FOCS/STOC-style conference with proceedings would help quantum information people get jobs, even in CS departments, and it might well hurt. In the current system, you can submit your paper to both QIP and FOCS/STOC and get accepted to both.”
Except that there is now added work of having to make a 3 page version of an already compressed FOCS/STOC version. And it looks like you are double dipping when you submit to both. If you have a seriously good paper that could appear in STOC/FOCS, as you say, you might also submit it to QIP. If it were a proper CS conference, you would have to think harder about it. QIP has chosen not to compete with these conferences. As Aram says, maybe this is actually good as it means that you don’t get a FOCS/STOC clone. But the disconnect between what the broader CS community values the conference as, and what the QIP community values it as, is very wide. Maybe if senior quantum people keep writing letters that point this out, things will actually change.
Btw, regarding the Gacs paper that Dave mentions: that is one of my absolute favorite results (just the result itself…I haven’t learned the proof). But, the abstract is a reasonable summary. Add some comments about why 1d seems to be hard (talk about equilibrium stat mech etc..) for intuition. Give the claims about storing also a nonzero density of information, etc… This gives quite a nice summary in 3 pages. Any PC member that would read that summary should (if he has any clue!) say “If this result is correct, it must be published! Ok now it is worth going through all 40 pages.”
Sure, one could just have the 40 page paper and with good intro to it, it would be enough, but writing the 3 page summary doesn’t seem that hard. One of the best results ever in physics is the renormalization group. And there is a 4 page (maybe less) PRL on it, “Critical Phenomena in 3.99 Dimensions”, basically an extended abstract claiming “yeah, we solved it” I think this is the tradition that is being mimiced; a PRL does not have to be a short result, it can be a short paper introducing a very important long result.
The funny thing about the Gacs paper is that it WAS a 10 pager for FOCS at one point: http://www.cs.bu.edu/faculty/gacs/papers/focs97.ps.gz As Larry Gray says of this paper: “Fortunately, Gacs did not give up on finding a counterexample. Starting with the highly informal ideas of Kurdyumov, he eventually produced the paper[2], which presumably contains the first counterexample to the Positive Rates Conjecture. I say ‘presumably’ because I believe there is no one else in the world besides Gacs who understands that paper. When the preprint of it first became available in 1984, I was visiting UCLA. Rick Durrett and I tried unsuccessfully for about a month to read it….”
Here’s an idea. It seems like there is reasonable support for the 3 page abstract (except from Dave). Why not publish a special edition of QIC, as matt suggests, but instead of publishing the papers it just publishes the 3 page abstracts? That way everyone is free to submit the full version to FOCS/STOC if they want to increase their odds of someday getting a tenure-track position in a CS department, or to PRL and Nature Physics if they want to go to a physics department.
In order for this to gain any currency, we would have to actually cite the short QIC version in the future. Since most journals don’t have limits on references (even PRL lets your references run over the length limit now), this shouldn’t be a problem. And if you’re writing papers in Nature or Science where they actually do limit the number of references, well, you’re probably not worrying about jobs anyway. 🙂
One further thing is that this might give a justification to requiring an arxiv version at the time of submission to QIP so that the long version can be properly cited in the 3 page abstract. As I said in my first comment above, I think this is important because some people are abusing the trust of the PC from last year and still haven’t put a paper on the arxiv with all the details. *cough*aramandfernando*cough*.
You are trying to agree with everyone, like John Sidles does. It won’t work.
If you have to make something publication-quality, then it takes more work. But outsiders don’t notice these fine-grained differences in prestige. A physicist won’t respect a journal they haven’t heard of, and a computer scientist won’t respect a conference they haven’t heard of. So we’ll work more without increasing our external visibility.
Also, I hate to see a coauthor go unacknowledged. It’s not only Fernando and I who are being delinquent, but also Michal is being a slacker.
I’m not sure I see your point Aram. We have to put together the 3 page abstracts anyway. Now suppose the people whose talks are actually accepted to QIP have to make sure that the abstracts are publication worthy. If the abstract was good enough that the talk was accepted, then I can’t imagine it will take that much more work. The authors get compensated for that work by having a publication in a journal that will have approximately as many citations as the long version of the paper itself. And that long version can be published in FOCS/STOC/PRL/Nat. Phys. where people outside the community will recognize the quality of the work. (This quasi-double-publish is already what people do with PRL and PR(A-E), for example.)
It seems to me that our external visibility is going strictly up, since we have more papers and citations (things the external world can objectively measure), and the workload is negligible. We don’t skimp on external prestige because we still publish the long version in the traditional powerhouse conferences/journals. Moreover, it raises the profile of QI and the whole community because we are only doing this with the top 20% of papers in the field, and I think people will quickly begin to see that level of quality and associate it to QIP once they start citing these works.
I guess it’s only work for the accepted talks, so that’s not so bad. It’s doing a lot of work for something with a 20% acceptance rate that’s more painful.
So I take back my objection.
When I was QIP PC chair, we posted the accepted submissions on the conference website. Before doing so, however, we checked with someone associated with STOC or FOCS (maybe some ACM people?) in order to verify that it wouldn’t rule out submissions to those conferences. They were OK with posting to the conference website, but seemed leary about anything more. I think particularly anything designed to make a QIP submission more like a regular paper, counting as a separate publication, might well result in having QIP ruled a separate publication venue and bar a simultaneous or subsequent submission to a standard CS conference.
@Daniel, I guess that would seem to rule out my idea. Oh well.
“You are trying to agree with everyone”
Hey. I’m part of everyone.
I hate to break it to you Steve, but having a few Science papers isn’t a silver bullet for the jobs problem.
I’ve been thinking a bit about my only half serious question about why we use program committees (and the people they rope in to to help) to select papers in the first place. So some thoughts: the amount of time people spend making 10 page, 3 page, ‘synopsis’, etc seems like misplaced labor. The arXiv is a defacto place where people put their work, in a form that they feel confortable sharing it with the rest of the world. So why shouldn’t the arxiv be the place for submission.
Now lets say all papers submitted go to the arxiv. You would submit your paper to a conference by just giving the arxiv id of the paper. So now who has to do the hard work of reading them? Well suppose you had a system which allowed all of the last years attendees to rate the papers. And here I think you could do this, wide open, in the public. And you could also add the ability for people, including the authors of the paper, to post comments on the paper. If you feel like your paper needs a 3 page synopsis in order to sweet talk people into giving a talk, you could put it there. If you prove BQP does not equal BPP, well you might forgo that because your comfortable with the paper standing on it’s own.
Then how is the cutoff made? Well here things could get interesting. You could just rank the papers by average ranking. But you could also do something like include X amount by ranking, and then Y amount by the most controversial papers. I think this latter type of adjustment is needed in order to keep a conference on its toes and subject to outside disruption.
Half baked. Thoughts?
This has a lot going for it, but how do you get people to concentrate their attention? The advantage of being a referee, or being on a PC, is precisely that you know that if you don’t do a good job reading an article then you can’t count on anyone else doing it. If everything is crowdsourced, then we risk not having people do a careful job looking for bugs, or for previous work that already contains the results.
Now, you could argue that finding bugs is not so important, and that it’s better to maximize community interest in a topic. Or you could say that a lot of refereeing now is not that good, and that the shame isn’t a good enough motivator. Both would be reasonable positions. But there are definitely tradeoffs.
Suppose you allow anonymous submissions for “bugs”…. Bad for people like me with big mouths cus surely there are many gunning for you 🙂
Here is a serious question: how many times do conference’s catch problems with papers. My experience has been that most problems of this form are for the pretty far extremes for paper that simply stand no chance of being accepted.
Perhaps there could be a hybrid system: Have some crowd-sourced voting that nominates roughly 2 times the number of submissions that will eventually be accepted. Then actually carefully referee those nominated submissions, also allowing some sort of scirate-like open discussion.
Well the fire seems to have died down. I think QIP is a great conference in how it has served up an awfully good amount of quality research. But why has this not been accompanied by a rise in its reputation across larger swaths of computer science. FOCS/STOC, too, were once obscure conferences. What is it that makes one conference take on more importance than others? (And of course, is it worth worrying about? No, probably not.)
This is a bit of a chicken-and-egg question. The way I see it, CS departments (in N. America at least) accept that theory is an important part of the field, and that they should devote faculty slots and resources to that. Then the CS theory community in turn use FOCS/STOC and other inward-facing mechanisms to decide who to try to hire, and to decide on where the boundaries of their field are (e.g. at some point, they decided that it included algorithmic game theory). In this picture, the nature of FOCS/STOC (e.g. whether there are published proceedings, or whether having a talk accepted is competitive) seems less important.
I think the real question to ask is why CS departments don’t want to hire quantum people. One possibility is that our talks and papers are hard for outsiders to understand, compared to any other area in CS. (Whereas in physics, everyone accepts that you need background to understand different subtopics.) But that can’t be the whole story.