We had a Mystery Hunt
Jan. 22nd, 2011 10:04 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
dr_whomVarious thoughts and reflections on the Hunt just past, not really organized.
I totally wasn't expecting to be elected one of the writing team's co-captains, by the way. I wasn't even intending to be a candidate. Erin C nominated me, and in her nomination e-mail she complimented my "dedication and enthusiasm"—and as some of you guys reading this are probably aware, those are sort of magic words for me and I couldn't turn it down. I'm still extremely flattered to have been elected, and then the Hunt proceeded to totally eat my life for the next twelve months.
If you asked me what my biggest mistake was in terms of the general writing process, my answer would probably be spending too much time on writing metas. Obviously we had a large number of metas in the Hunt, but just "writing a lot of metas" isn't what I mean here; the mistake is that we wrote too many more metas than we needed. Because of the "minihunt" concept, where the Hunt was going to consist of four-to-six gameworlds each with its own distinctive meta structure, the plan was to brainstorm a bunch of good proposals for gameworlds and then vote on which ones we wanted to pin into the Hunt. But since most of the gameworld proposals were tied up with very specific metapuzzle concepts, that meant that we couldn't vote to pin a gameworld into the Hunt until its key metas were fully written and testsolved. For some, this wasn't too bad—for Mega Man, really we only needed confirmation of the supermeta before voting it into the Hunt, and most of the metas for the individual robots were written individually after the world structure itself had already been pinned in. On the other hand, all six metas and the supermeta for Civilization had to be fully testsolved before we could be willing to vote it into the Hunt, since we weren't willing to pin in a round with the concept "metas with densely overlapping puzzle sets in a tech tree" without some verification that we could actually write solvable metas fitting these constraints. But the consequences of that were that we didn't vote the last minihunts in until August 12th, and we used up quite a lot of writing, editing, and testsolving energy, which could have been devoted to other puzzles, on two entire gameworld structures and sets of metas that didn't wind up in the Hunt. (It would have been three, but at least Street Fighter found a home in the "Meta testing!" puzzle.) If I'd imposed an earlier moratorium on new gameworld proposals, we would have had a lot more breathing room at the end.
By Friday evening I was pretty sure the Hunt was going to be a disaster—or at least, that it would end up getting a judgment of "structurally innovative but really buggy and messy", as![[livejournal.com profile]](https://www.dreamwidth.org/img/external/lj-userinfo.gif)
okosut suggested. As late as 2am Friday we were still getting reports from proofreaders that some puzzles in the Mario rounds had gone onto the Hunt server in an obsolete version that didn't take into account revisions that had been made in response to testsolving. By kickoff only the Mario and Mega Man worlds had been fully proofread, and soon afterward bug reports started to come in on puzzles that teams had unlocked: "Karaoke Night" had an incorrect timestamp. "Rivalries" had an obsolete version of the grid. "N-tris" had a second consistent way of filling the grid that the author, editors, and testsolvers had never detected, and okosut had to revise it on the fly. Mega Man puzzles were being released in the wrong order. (Did anybody notice this?) Then "Funny Farm" crashed the server, and we had to call the teams that were working on it and ask them to stop, cancel the puzzle entirely, and replace it with our only backup puzzle. (I think it's a lucky coincidence that we had "coins" already built into our event system: they provided a reasonable way of compensating teams for time spent working on a puzzle that was cut mid-Hunt.) Soon a lot of teams started needing not only welcome-to-Mega-Man visits but also deliveries of Redundant Obsolescence and Powder Monkey, which was something else to keep track of. And all this time okosut and I had been kind of freaking out about what to do about the Princess Letter for the Civilization metameta, which I'll say more about in a later post. I just felt things seeming to spiral out of control, perhaps especially between when Andrew and Kate went to bed and when jcberk showed up.
And then... they got back into control. I'm not sure how exactly. We got past the flashpoint where a lot of teams needed physical puzzles delivered at the same time. More of the error reports turned out to be mistaken: a careful recount confirmed that the numbers on the last PbN grid in "Pointillisme" were consistent; a long meeting between authors and testsolvers concluded that what teams were reporting as an ambiguity in "Scrambling Attributes Yields Conundrum" was actually just a subtlety in the puzzle they had missed. It got late, and solver activity slowed down; we started asking people to come to us to pick up their white powders and so on. Cindy took over delegating copyediting of the Katamari round. And things just started going a whole lot more smoothly. It looked for a while as if three teams were going to be starting on endgame pretty close to each other, and we had a big frantic meeting on what we'd do if that happened (again), but then Codex solved six metas in an hour and a half and sailed into endgame, and the potential choke point of the final runaround remained unclogged. So I was really worried for a while there on Friday evening, but in the end it seems as if the Hunt turned out okay and fairly well-liked.
On Saturday of the Hunt I was able to visit a few teams—this was between when I was worrying about copyediting puzzles and when I started worry about copyediting solutions, and Kate and Andrew were back at HQ so I didn't have to be present to be in charge of running everything, either. Setec, Codex, and IIF all seemed to be mostly having a pretty good time. I do feel bad that I didn't get a chance to visit any smaller teams, to see a different side of the Hunt, or Manic Sages (though I talked to them on the phone a lot).
One deliberate feature of the Hunt's structure was a higher-than-normal ratio of metas to other puzzles—both because it made for interesting structures at the size of about a fifth of the Hunt, and because hey, we like metas. A possibly unforeseen (well, by me, anyway), though certainly foreseeable, consequence was that a lot of the harder puzzles got fairly few solves even from the stronger teams—it was too easy to just skip them or backsolve around them, since there was so much data floating around from the metas. And I mean, you'll find few bigger fans of backsolving than me (although this guy probably counts), but I feel kind of bad for those puzzles; a lot of them were pretty awesome and it's sad to see solvers missing out on them in that sense. On the other hand, I wouldn't want to put in arbitrary bottlenecks just to make people work on my favorite puzzles; and indeed, the coin was found at a perfectly reasonable time, so I can't even say the puzzles that teams did work on should have been harder than they were. So I'm not sure what the right way to thread this needle would have been.
One of my main goals in editing this Hunt was puzzle thematicity—i.e., having puzzles that have some connection to the theme of the round they're in and/or to their answer. I had an uphill battle here on two counts: One, in order to preserve the integrity of testsolving, most potential puzzle authors couldn't view the list of available puzzle answers, and therefore usually couldn't use answers as a way of brainstorming puzzle concepts that fit them. And two, often the best place to tie a puzzle in thematically with its answer and/or round is in the flavortext, and a lot of the other puzzle editors on Metaphysical—hinoahspuzzlelj! hi Chris!—have this vendetta against flavortext that I don't really understand. (I mean, I know people have gotten led seriously astray occasionally by chasing red herrings from flavortext, but I think that's an overreaction. Flavortext certainly can be done right; consider the 2010 Hunt, which was fairly clean and red-herring–free in this regard, and had very rich flavortext.) I regarded puzzle thematicity as most important in the Mega Man world (where each robot had its own distinct theme) and the Civ world (where each puzzle had its own tech name), and I think in those two we did an okay job with thematicity, though there are still some clunkers in this respect (an MIT textbooks puzzle in a gambling-themed round? I guess we probably could have come up with some way to justify it if we'd tried, but there's nothing obvious to use, and we didn't really try). So overall I'd say that this goal of mine met with mixed success as far as puzzles are concerned. It worked extremely well for metas, though, since everyone seemed to agree that thematicity of metas was a goal; compare SPIES, in which only one of nine metas had any particular thematic relationship to the city it was supposedly set in (though the answers were thematic to the spies themselves, to be fair).
I continue to be impressed by all the random awesome stuff we pulled off that we didn't even have to do. The room in which teams met GLaDOS during endgame was spectacular, and creepy in just the right way, and Allen and Joel and Roger went to a lot of effort to throw it together—and it didn't even have any puzzles in it! Not only did we get Peter to write a Mario-themed pastiche of the Mendelssohn wedding march (which, I mean, I'm sure he would have done anyway), we actually got members of our team to rehearse it and perform it at kickoff! The parody of "Still Alive" and closing credits! The achievements! That Bowser costume! And so forth: a lot of my teammates really went above and beyond the call of duty in doing stuff that, if we didn't have them, nobody would have even noticed—but having them made our Hunt a much more awesome experience.
(In retrospect, though, I'm not sure the congratulations e-mails we sent out whenever a team solved a supermeta were worth doing if we didn't have time to automate them. I kept being terrified that I would send a congratulations note to the wrong team, or for the wrong supermeta. I don't think I did, but it was one more source of anxiety than I needed.)
In a future post or two, I'll make comments on specific puzzles!
I totally wasn't expecting to be elected one of the writing team's co-captains, by the way. I wasn't even intending to be a candidate. Erin C nominated me, and in her nomination e-mail she complimented my "dedication and enthusiasm"—and as some of you guys reading this are probably aware, those are sort of magic words for me and I couldn't turn it down. I'm still extremely flattered to have been elected, and then the Hunt proceeded to totally eat my life for the next twelve months.
If you asked me what my biggest mistake was in terms of the general writing process, my answer would probably be spending too much time on writing metas. Obviously we had a large number of metas in the Hunt, but just "writing a lot of metas" isn't what I mean here; the mistake is that we wrote too many more metas than we needed. Because of the "minihunt" concept, where the Hunt was going to consist of four-to-six gameworlds each with its own distinctive meta structure, the plan was to brainstorm a bunch of good proposals for gameworlds and then vote on which ones we wanted to pin into the Hunt. But since most of the gameworld proposals were tied up with very specific metapuzzle concepts, that meant that we couldn't vote to pin a gameworld into the Hunt until its key metas were fully written and testsolved. For some, this wasn't too bad—for Mega Man, really we only needed confirmation of the supermeta before voting it into the Hunt, and most of the metas for the individual robots were written individually after the world structure itself had already been pinned in. On the other hand, all six metas and the supermeta for Civilization had to be fully testsolved before we could be willing to vote it into the Hunt, since we weren't willing to pin in a round with the concept "metas with densely overlapping puzzle sets in a tech tree" without some verification that we could actually write solvable metas fitting these constraints. But the consequences of that were that we didn't vote the last minihunts in until August 12th, and we used up quite a lot of writing, editing, and testsolving energy, which could have been devoted to other puzzles, on two entire gameworld structures and sets of metas that didn't wind up in the Hunt. (It would have been three, but at least Street Fighter found a home in the "Meta testing!" puzzle.) If I'd imposed an earlier moratorium on new gameworld proposals, we would have had a lot more breathing room at the end.
By Friday evening I was pretty sure the Hunt was going to be a disaster—or at least, that it would end up getting a judgment of "structurally innovative but really buggy and messy", as
okosut suggested. As late as 2am Friday we were still getting reports from proofreaders that some puzzles in the Mario rounds had gone onto the Hunt server in an obsolete version that didn't take into account revisions that had been made in response to testsolving. By kickoff only the Mario and Mega Man worlds had been fully proofread, and soon afterward bug reports started to come in on puzzles that teams had unlocked: "Karaoke Night" had an incorrect timestamp. "Rivalries" had an obsolete version of the grid. "N-tris" had a second consistent way of filling the grid that the author, editors, and testsolvers had never detected, and okosut had to revise it on the fly. Mega Man puzzles were being released in the wrong order. (Did anybody notice this?) Then "Funny Farm" crashed the server, and we had to call the teams that were working on it and ask them to stop, cancel the puzzle entirely, and replace it with our only backup puzzle. (I think it's a lucky coincidence that we had "coins" already built into our event system: they provided a reasonable way of compensating teams for time spent working on a puzzle that was cut mid-Hunt.) Soon a lot of teams started needing not only welcome-to-Mega-Man visits but also deliveries of Redundant Obsolescence and Powder Monkey, which was something else to keep track of. And all this time okosut and I had been kind of freaking out about what to do about the Princess Letter for the Civilization metameta, which I'll say more about in a later post. I just felt things seeming to spiral out of control, perhaps especially between when Andrew and Kate went to bed and when jcberk showed up.And then... they got back into control. I'm not sure how exactly. We got past the flashpoint where a lot of teams needed physical puzzles delivered at the same time. More of the error reports turned out to be mistaken: a careful recount confirmed that the numbers on the last PbN grid in "Pointillisme" were consistent; a long meeting between authors and testsolvers concluded that what teams were reporting as an ambiguity in "Scrambling Attributes Yields Conundrum" was actually just a subtlety in the puzzle they had missed. It got late, and solver activity slowed down; we started asking people to come to us to pick up their white powders and so on. Cindy took over delegating copyediting of the Katamari round. And things just started going a whole lot more smoothly. It looked for a while as if three teams were going to be starting on endgame pretty close to each other, and we had a big frantic meeting on what we'd do if that happened (again), but then Codex solved six metas in an hour and a half and sailed into endgame, and the potential choke point of the final runaround remained unclogged. So I was really worried for a while there on Friday evening, but in the end it seems as if the Hunt turned out okay and fairly well-liked.
On Saturday of the Hunt I was able to visit a few teams—this was between when I was worrying about copyediting puzzles and when I started worry about copyediting solutions, and Kate and Andrew were back at HQ so I didn't have to be present to be in charge of running everything, either. Setec, Codex, and IIF all seemed to be mostly having a pretty good time. I do feel bad that I didn't get a chance to visit any smaller teams, to see a different side of the Hunt, or Manic Sages (though I talked to them on the phone a lot).
One deliberate feature of the Hunt's structure was a higher-than-normal ratio of metas to other puzzles—both because it made for interesting structures at the size of about a fifth of the Hunt, and because hey, we like metas. A possibly unforeseen (well, by me, anyway), though certainly foreseeable, consequence was that a lot of the harder puzzles got fairly few solves even from the stronger teams—it was too easy to just skip them or backsolve around them, since there was so much data floating around from the metas. And I mean, you'll find few bigger fans of backsolving than me (although this guy probably counts), but I feel kind of bad for those puzzles; a lot of them were pretty awesome and it's sad to see solvers missing out on them in that sense. On the other hand, I wouldn't want to put in arbitrary bottlenecks just to make people work on my favorite puzzles; and indeed, the coin was found at a perfectly reasonable time, so I can't even say the puzzles that teams did work on should have been harder than they were. So I'm not sure what the right way to thread this needle would have been.
One of my main goals in editing this Hunt was puzzle thematicity—i.e., having puzzles that have some connection to the theme of the round they're in and/or to their answer. I had an uphill battle here on two counts: One, in order to preserve the integrity of testsolving, most potential puzzle authors couldn't view the list of available puzzle answers, and therefore usually couldn't use answers as a way of brainstorming puzzle concepts that fit them. And two, often the best place to tie a puzzle in thematically with its answer and/or round is in the flavortext, and a lot of the other puzzle editors on Metaphysical—hi
noahspuzzlelj! hi Chris!—have this vendetta against flavortext that I don't really understand. (I mean, I know people have gotten led seriously astray occasionally by chasing red herrings from flavortext, but I think that's an overreaction. Flavortext certainly can be done right; consider the 2010 Hunt, which was fairly clean and red-herring–free in this regard, and had very rich flavortext.) I regarded puzzle thematicity as most important in the Mega Man world (where each robot had its own distinct theme) and the Civ world (where each puzzle had its own tech name), and I think in those two we did an okay job with thematicity, though there are still some clunkers in this respect (an MIT textbooks puzzle in a gambling-themed round? I guess we probably could have come up with some way to justify it if we'd tried, but there's nothing obvious to use, and we didn't really try). So overall I'd say that this goal of mine met with mixed success as far as puzzles are concerned. It worked extremely well for metas, though, since everyone seemed to agree that thematicity of metas was a goal; compare SPIES, in which only one of nine metas had any particular thematic relationship to the city it was supposedly set in (though the answers were thematic to the spies themselves, to be fair).I continue to be impressed by all the random awesome stuff we pulled off that we didn't even have to do. The room in which teams met GLaDOS during endgame was spectacular, and creepy in just the right way, and Allen and Joel and Roger went to a lot of effort to throw it together—and it didn't even have any puzzles in it! Not only did we get Peter to write a Mario-themed pastiche of the Mendelssohn wedding march (which, I mean, I'm sure he would have done anyway), we actually got members of our team to rehearse it and perform it at kickoff! The parody of "Still Alive" and closing credits! The achievements! That Bowser costume! And so forth: a lot of my teammates really went above and beyond the call of duty in doing stuff that, if we didn't have them, nobody would have even noticed—but having them made our Hunt a much more awesome experience.
(In retrospect, though, I'm not sure the congratulations e-mails we sent out whenever a team solved a supermeta were worth doing if we didn't have time to automate them. I kept being terrified that I would send a congratulations note to the wrong team, or for the wrong supermeta. I don't think I did, but it was one more source of anxiety than I needed.)
In a future post or two, I'll make comments on specific puzzles!
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
no subject
Date: 2011-01-23 03:36 am (UTC)no subject
Date: 2011-01-23 03:37 am (UTC)I'd have really loved it if we had the time to pay that much attention to all the puzzles as we did to the metas. It'd be great if we could spend a month coming up with 5 great Stagecraft man puzzles. I loved the suggestion that all puzzles in Mario be named after Mario badguys. In a perfect world we'd have had an email list of people for each of the worlds who would write puzzles that really fit into that world. But if we gave every puzzle slot as much attention as the metas got we'd never finish a mystery hunt in a year.
So ironically I think it's largely due to the thing you consider our biggest mistake that we were able to be so successful on your goal as it concerned metas.
no subject
Date: 2011-01-23 05:16 am (UTC)(Also, somebody at one point definitely said something like 'I'm not that interested in making puzzles thematic to their rounds; the metas and round structures themselves are where the thematicity lies in our Hunt'?)
"Tying things in" is definitely where the core disagreement lies, sure.
no subject
Date: 2011-01-23 05:37 am (UTC)I wonder to what extent this sort of thing would be feasible. If we spent time brainstorming appropriate puzzle topics, would we actually be able to fill rounds entirely thematically? We did very tight puzzle theming in BANG 17; could this be replicated on a grander scale with more brains? Probably not without some serious delegation- giving editors different rounds/worlds to spearhead, etc.
no subject
Date: 2011-01-23 04:58 pm (UTC)no subject
Date: 2011-01-23 05:18 pm (UTC)We had some post-hoc tie-ins this year that I'm really proud of, by the way: Mountain Pass, Good Vibrations, Good Times in the Casino, Technological Crisis at Shikakuro Farms, Losing My Nerve, and a few others. None of those are puzzles we'd have thought of if we'd been deliberately trying to come up with puzzles that fit the themes of their rounds, but once we slotted them in there and wrote some flavortext they feel like totally natural fits in context.
no subject
Date: 2011-01-23 05:35 pm (UTC)Thematicity in Civ just would have meant things that actually genuinely matched an appropriate technology. This isn't too hard, but more importantly wouldn't result at all in a world where the puzzles felt the same. Similarly for KD we'd just want all the titles to be objects of an appropriate size. Again that wouldn't harm the puzzle experience the way that a round of sports puzzles does.
My experience from Iron Puzzler is that if you have 6 different groups all thinking about writing a puzzle based on X, they'll usually come up with 3 or 4 genuinely different takes.
no subject
Date: 2011-01-24 07:45 pm (UTC)I agree that given the time constraints, it would have been difficult to put as much care into theming all of our puzzles as we did the metas, and given the option, metas were certainly the right choice. But I wonder what we could have done to help with the other puzzles. We did have those brainstorming lists that were somewhat underutilized. That may have been due to a general malaise settling in at the time, but perhaps if someone had said, "we'd like you guys to spend some time brainstorming the following puzzles: a Stagecraft Man puzzle with answer FOO, a Blackberry Man puzzle with answer BAR, etc.," people would have been more motivated to talk specifics.
-Ricky
no subject
Date: 2011-01-25 05:30 am (UTC)no subject
Date: 2011-01-23 04:13 am (UTC)Which is a way of saying the music and Bowser costume and the rest of it were things that we did notice, and appreciated. The final-credits roll, for example? Beautiful way to make us feel cared about as an audience, and I write as someone who never saw Portal before this weekend.
no subject
Date: 2011-01-23 05:02 am (UTC)no subject
Date: 2011-01-23 04:43 am (UTC)* Don't write more metas than you need
* Try to proofread before the Hunt starts
* Try not to have to run things to teams all at the same time
Will do! (Hopefully...) :-)
no subject
Date: 2011-01-23 05:01 am (UTC)Also, as per
* Have a backup puzzle ready to stick in in case some other puzzle unexpectedly implodes.
We actually had another couple of backups we could potentially have used if another puzzle had imploded before any team unlocked the first round of Katamari. Not floaters, but puzzles that testsolvers had given kind of blah ratings to that we'd cut in favor of floaters. If we'd had to, we could have put one of them back in and moved the floaters that had replaced them to an unexpectedly vacated slot.
no subject
Date: 2011-01-23 05:30 am (UTC)no subject
Date: 2011-01-23 05:08 pm (UTC)Another thing to keep in mind for running teams is balancing the kinds of energy in the core group. I think mentioning Jen showing up is not a coincidence, which is to say she's calm and unflappable while Aaron you're a little more prone to worrying. Similarly, Andrew is a good calming influence when he's around. Of course, some amount of worrying is crucial because it's part of what finds the actual problems. But it's important to make sure that you've got the right balance.
no subject
Date: 2011-01-23 07:48 pm (UTC)I'll tell my favorite last-minute-proofreading story from SPIES. In Kuala Lampur (http://web.mit.edu/puzzle/www/06/puzzles/kuala_lumpur/), every puzzle was associated with a skyscraper, with a labeled height. These were the actual names of the 12 tallest buildings in Kuala Lampur, and their actual heights. Sometime later, our art team decided to obtain photos of these 12 buildings, and to do drawings of them for the round's front page. Someone else, assembling the final graphics for the round, was unaware that these were real buildings, and grouped together the building names, the heights, the photos, and the drawings at random.
This was caught a few hours before the leading teams made it to KL, and Jeff F. and I unscrambled everything before they got there. If we hadn't, teams would have had an immensely challenging puzzle in figuring out what the incorrect groupings meant!
Again, I'm not sure what the best way to prevent these errors is, but I will definitely warn that, without a mechanism in place, they will pop up everywhere.
David S.
no subject
Date: 2011-01-23 07:57 pm (UTC)I'm hoping the software we end up using supports a plan like this; as far as I know our plan is to use the MetaFungus software with as few alterations as possible, but I haven't actually laid eyes on said software yet.
no subject
Date: 2011-01-24 04:45 am (UTC)2011 Hunt roundup
Date: 2011-01-23 08:48 am (UTC)