We had a Mystery Hunt
Jan. 22nd, 2011 10:04 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
dr_whomVarious thoughts and reflections on the Hunt just past, not really organized.
I totally wasn't expecting to be elected one of the writing team's co-captains, by the way. I wasn't even intending to be a candidate. Erin C nominated me, and in her nomination e-mail she complimented my "dedication and enthusiasm"—and as some of you guys reading this are probably aware, those are sort of magic words for me and I couldn't turn it down. I'm still extremely flattered to have been elected, and then the Hunt proceeded to totally eat my life for the next twelve months.
If you asked me what my biggest mistake was in terms of the general writing process, my answer would probably be spending too much time on writing metas. Obviously we had a large number of metas in the Hunt, but just "writing a lot of metas" isn't what I mean here; the mistake is that we wrote too many more metas than we needed. Because of the "minihunt" concept, where the Hunt was going to consist of four-to-six gameworlds each with its own distinctive meta structure, the plan was to brainstorm a bunch of good proposals for gameworlds and then vote on which ones we wanted to pin into the Hunt. But since most of the gameworld proposals were tied up with very specific metapuzzle concepts, that meant that we couldn't vote to pin a gameworld into the Hunt until its key metas were fully written and testsolved. For some, this wasn't too bad—for Mega Man, really we only needed confirmation of the supermeta before voting it into the Hunt, and most of the metas for the individual robots were written individually after the world structure itself had already been pinned in. On the other hand, all six metas and the supermeta for Civilization had to be fully testsolved before we could be willing to vote it into the Hunt, since we weren't willing to pin in a round with the concept "metas with densely overlapping puzzle sets in a tech tree" without some verification that we could actually write solvable metas fitting these constraints. But the consequences of that were that we didn't vote the last minihunts in until August 12th, and we used up quite a lot of writing, editing, and testsolving energy, which could have been devoted to other puzzles, on two entire gameworld structures and sets of metas that didn't wind up in the Hunt. (It would have been three, but at least Street Fighter found a home in the "Meta testing!" puzzle.) If I'd imposed an earlier moratorium on new gameworld proposals, we would have had a lot more breathing room at the end.
By Friday evening I was pretty sure the Hunt was going to be a disaster—or at least, that it would end up getting a judgment of "structurally innovative but really buggy and messy", as![[livejournal.com profile]](https://www.dreamwidth.org/img/external/lj-userinfo.gif)
okosut suggested. As late as 2am Friday we were still getting reports from proofreaders that some puzzles in the Mario rounds had gone onto the Hunt server in an obsolete version that didn't take into account revisions that had been made in response to testsolving. By kickoff only the Mario and Mega Man worlds had been fully proofread, and soon afterward bug reports started to come in on puzzles that teams had unlocked: "Karaoke Night" had an incorrect timestamp. "Rivalries" had an obsolete version of the grid. "N-tris" had a second consistent way of filling the grid that the author, editors, and testsolvers had never detected, and okosut had to revise it on the fly. Mega Man puzzles were being released in the wrong order. (Did anybody notice this?) Then "Funny Farm" crashed the server, and we had to call the teams that were working on it and ask them to stop, cancel the puzzle entirely, and replace it with our only backup puzzle. (I think it's a lucky coincidence that we had "coins" already built into our event system: they provided a reasonable way of compensating teams for time spent working on a puzzle that was cut mid-Hunt.) Soon a lot of teams started needing not only welcome-to-Mega-Man visits but also deliveries of Redundant Obsolescence and Powder Monkey, which was something else to keep track of. And all this time okosut and I had been kind of freaking out about what to do about the Princess Letter for the Civilization metameta, which I'll say more about in a later post. I just felt things seeming to spiral out of control, perhaps especially between when Andrew and Kate went to bed and when jcberk showed up.
And then... they got back into control. I'm not sure how exactly. We got past the flashpoint where a lot of teams needed physical puzzles delivered at the same time. More of the error reports turned out to be mistaken: a careful recount confirmed that the numbers on the last PbN grid in "Pointillisme" were consistent; a long meeting between authors and testsolvers concluded that what teams were reporting as an ambiguity in "Scrambling Attributes Yields Conundrum" was actually just a subtlety in the puzzle they had missed. It got late, and solver activity slowed down; we started asking people to come to us to pick up their white powders and so on. Cindy took over delegating copyediting of the Katamari round. And things just started going a whole lot more smoothly. It looked for a while as if three teams were going to be starting on endgame pretty close to each other, and we had a big frantic meeting on what we'd do if that happened (again), but then Codex solved six metas in an hour and a half and sailed into endgame, and the potential choke point of the final runaround remained unclogged. So I was really worried for a while there on Friday evening, but in the end it seems as if the Hunt turned out okay and fairly well-liked.
On Saturday of the Hunt I was able to visit a few teams—this was between when I was worrying about copyediting puzzles and when I started worry about copyediting solutions, and Kate and Andrew were back at HQ so I didn't have to be present to be in charge of running everything, either. Setec, Codex, and IIF all seemed to be mostly having a pretty good time. I do feel bad that I didn't get a chance to visit any smaller teams, to see a different side of the Hunt, or Manic Sages (though I talked to them on the phone a lot).
One deliberate feature of the Hunt's structure was a higher-than-normal ratio of metas to other puzzles—both because it made for interesting structures at the size of about a fifth of the Hunt, and because hey, we like metas. A possibly unforeseen (well, by me, anyway), though certainly foreseeable, consequence was that a lot of the harder puzzles got fairly few solves even from the stronger teams—it was too easy to just skip them or backsolve around them, since there was so much data floating around from the metas. And I mean, you'll find few bigger fans of backsolving than me (although this guy probably counts), but I feel kind of bad for those puzzles; a lot of them were pretty awesome and it's sad to see solvers missing out on them in that sense. On the other hand, I wouldn't want to put in arbitrary bottlenecks just to make people work on my favorite puzzles; and indeed, the coin was found at a perfectly reasonable time, so I can't even say the puzzles that teams did work on should have been harder than they were. So I'm not sure what the right way to thread this needle would have been.
One of my main goals in editing this Hunt was puzzle thematicity—i.e., having puzzles that have some connection to the theme of the round they're in and/or to their answer. I had an uphill battle here on two counts: One, in order to preserve the integrity of testsolving, most potential puzzle authors couldn't view the list of available puzzle answers, and therefore usually couldn't use answers as a way of brainstorming puzzle concepts that fit them. And two, often the best place to tie a puzzle in thematically with its answer and/or round is in the flavortext, and a lot of the other puzzle editors on Metaphysical—hinoahspuzzlelj! hi Chris!—have this vendetta against flavortext that I don't really understand. (I mean, I know people have gotten led seriously astray occasionally by chasing red herrings from flavortext, but I think that's an overreaction. Flavortext certainly can be done right; consider the 2010 Hunt, which was fairly clean and red-herring–free in this regard, and had very rich flavortext.) I regarded puzzle thematicity as most important in the Mega Man world (where each robot had its own distinct theme) and the Civ world (where each puzzle had its own tech name), and I think in those two we did an okay job with thematicity, though there are still some clunkers in this respect (an MIT textbooks puzzle in a gambling-themed round? I guess we probably could have come up with some way to justify it if we'd tried, but there's nothing obvious to use, and we didn't really try). So overall I'd say that this goal of mine met with mixed success as far as puzzles are concerned. It worked extremely well for metas, though, since everyone seemed to agree that thematicity of metas was a goal; compare SPIES, in which only one of nine metas had any particular thematic relationship to the city it was supposedly set in (though the answers were thematic to the spies themselves, to be fair).
I continue to be impressed by all the random awesome stuff we pulled off that we didn't even have to do. The room in which teams met GLaDOS during endgame was spectacular, and creepy in just the right way, and Allen and Joel and Roger went to a lot of effort to throw it together—and it didn't even have any puzzles in it! Not only did we get Peter to write a Mario-themed pastiche of the Mendelssohn wedding march (which, I mean, I'm sure he would have done anyway), we actually got members of our team to rehearse it and perform it at kickoff! The parody of "Still Alive" and closing credits! The achievements! That Bowser costume! And so forth: a lot of my teammates really went above and beyond the call of duty in doing stuff that, if we didn't have them, nobody would have even noticed—but having them made our Hunt a much more awesome experience.
(In retrospect, though, I'm not sure the congratulations e-mails we sent out whenever a team solved a supermeta were worth doing if we didn't have time to automate them. I kept being terrified that I would send a congratulations note to the wrong team, or for the wrong supermeta. I don't think I did, but it was one more source of anxiety than I needed.)
In a future post or two, I'll make comments on specific puzzles!
I totally wasn't expecting to be elected one of the writing team's co-captains, by the way. I wasn't even intending to be a candidate. Erin C nominated me, and in her nomination e-mail she complimented my "dedication and enthusiasm"—and as some of you guys reading this are probably aware, those are sort of magic words for me and I couldn't turn it down. I'm still extremely flattered to have been elected, and then the Hunt proceeded to totally eat my life for the next twelve months.
If you asked me what my biggest mistake was in terms of the general writing process, my answer would probably be spending too much time on writing metas. Obviously we had a large number of metas in the Hunt, but just "writing a lot of metas" isn't what I mean here; the mistake is that we wrote too many more metas than we needed. Because of the "minihunt" concept, where the Hunt was going to consist of four-to-six gameworlds each with its own distinctive meta structure, the plan was to brainstorm a bunch of good proposals for gameworlds and then vote on which ones we wanted to pin into the Hunt. But since most of the gameworld proposals were tied up with very specific metapuzzle concepts, that meant that we couldn't vote to pin a gameworld into the Hunt until its key metas were fully written and testsolved. For some, this wasn't too bad—for Mega Man, really we only needed confirmation of the supermeta before voting it into the Hunt, and most of the metas for the individual robots were written individually after the world structure itself had already been pinned in. On the other hand, all six metas and the supermeta for Civilization had to be fully testsolved before we could be willing to vote it into the Hunt, since we weren't willing to pin in a round with the concept "metas with densely overlapping puzzle sets in a tech tree" without some verification that we could actually write solvable metas fitting these constraints. But the consequences of that were that we didn't vote the last minihunts in until August 12th, and we used up quite a lot of writing, editing, and testsolving energy, which could have been devoted to other puzzles, on two entire gameworld structures and sets of metas that didn't wind up in the Hunt. (It would have been three, but at least Street Fighter found a home in the "Meta testing!" puzzle.) If I'd imposed an earlier moratorium on new gameworld proposals, we would have had a lot more breathing room at the end.
By Friday evening I was pretty sure the Hunt was going to be a disaster—or at least, that it would end up getting a judgment of "structurally innovative but really buggy and messy", as
okosut suggested. As late as 2am Friday we were still getting reports from proofreaders that some puzzles in the Mario rounds had gone onto the Hunt server in an obsolete version that didn't take into account revisions that had been made in response to testsolving. By kickoff only the Mario and Mega Man worlds had been fully proofread, and soon afterward bug reports started to come in on puzzles that teams had unlocked: "Karaoke Night" had an incorrect timestamp. "Rivalries" had an obsolete version of the grid. "N-tris" had a second consistent way of filling the grid that the author, editors, and testsolvers had never detected, and okosut had to revise it on the fly. Mega Man puzzles were being released in the wrong order. (Did anybody notice this?) Then "Funny Farm" crashed the server, and we had to call the teams that were working on it and ask them to stop, cancel the puzzle entirely, and replace it with our only backup puzzle. (I think it's a lucky coincidence that we had "coins" already built into our event system: they provided a reasonable way of compensating teams for time spent working on a puzzle that was cut mid-Hunt.) Soon a lot of teams started needing not only welcome-to-Mega-Man visits but also deliveries of Redundant Obsolescence and Powder Monkey, which was something else to keep track of. And all this time okosut and I had been kind of freaking out about what to do about the Princess Letter for the Civilization metameta, which I'll say more about in a later post. I just felt things seeming to spiral out of control, perhaps especially between when Andrew and Kate went to bed and when jcberk showed up.And then... they got back into control. I'm not sure how exactly. We got past the flashpoint where a lot of teams needed physical puzzles delivered at the same time. More of the error reports turned out to be mistaken: a careful recount confirmed that the numbers on the last PbN grid in "Pointillisme" were consistent; a long meeting between authors and testsolvers concluded that what teams were reporting as an ambiguity in "Scrambling Attributes Yields Conundrum" was actually just a subtlety in the puzzle they had missed. It got late, and solver activity slowed down; we started asking people to come to us to pick up their white powders and so on. Cindy took over delegating copyediting of the Katamari round. And things just started going a whole lot more smoothly. It looked for a while as if three teams were going to be starting on endgame pretty close to each other, and we had a big frantic meeting on what we'd do if that happened (again), but then Codex solved six metas in an hour and a half and sailed into endgame, and the potential choke point of the final runaround remained unclogged. So I was really worried for a while there on Friday evening, but in the end it seems as if the Hunt turned out okay and fairly well-liked.
On Saturday of the Hunt I was able to visit a few teams—this was between when I was worrying about copyediting puzzles and when I started worry about copyediting solutions, and Kate and Andrew were back at HQ so I didn't have to be present to be in charge of running everything, either. Setec, Codex, and IIF all seemed to be mostly having a pretty good time. I do feel bad that I didn't get a chance to visit any smaller teams, to see a different side of the Hunt, or Manic Sages (though I talked to them on the phone a lot).
One deliberate feature of the Hunt's structure was a higher-than-normal ratio of metas to other puzzles—both because it made for interesting structures at the size of about a fifth of the Hunt, and because hey, we like metas. A possibly unforeseen (well, by me, anyway), though certainly foreseeable, consequence was that a lot of the harder puzzles got fairly few solves even from the stronger teams—it was too easy to just skip them or backsolve around them, since there was so much data floating around from the metas. And I mean, you'll find few bigger fans of backsolving than me (although this guy probably counts), but I feel kind of bad for those puzzles; a lot of them were pretty awesome and it's sad to see solvers missing out on them in that sense. On the other hand, I wouldn't want to put in arbitrary bottlenecks just to make people work on my favorite puzzles; and indeed, the coin was found at a perfectly reasonable time, so I can't even say the puzzles that teams did work on should have been harder than they were. So I'm not sure what the right way to thread this needle would have been.
One of my main goals in editing this Hunt was puzzle thematicity—i.e., having puzzles that have some connection to the theme of the round they're in and/or to their answer. I had an uphill battle here on two counts: One, in order to preserve the integrity of testsolving, most potential puzzle authors couldn't view the list of available puzzle answers, and therefore usually couldn't use answers as a way of brainstorming puzzle concepts that fit them. And two, often the best place to tie a puzzle in thematically with its answer and/or round is in the flavortext, and a lot of the other puzzle editors on Metaphysical—hi
noahspuzzlelj! hi Chris!—have this vendetta against flavortext that I don't really understand. (I mean, I know people have gotten led seriously astray occasionally by chasing red herrings from flavortext, but I think that's an overreaction. Flavortext certainly can be done right; consider the 2010 Hunt, which was fairly clean and red-herring–free in this regard, and had very rich flavortext.) I regarded puzzle thematicity as most important in the Mega Man world (where each robot had its own distinct theme) and the Civ world (where each puzzle had its own tech name), and I think in those two we did an okay job with thematicity, though there are still some clunkers in this respect (an MIT textbooks puzzle in a gambling-themed round? I guess we probably could have come up with some way to justify it if we'd tried, but there's nothing obvious to use, and we didn't really try). So overall I'd say that this goal of mine met with mixed success as far as puzzles are concerned. It worked extremely well for metas, though, since everyone seemed to agree that thematicity of metas was a goal; compare SPIES, in which only one of nine metas had any particular thematic relationship to the city it was supposedly set in (though the answers were thematic to the spies themselves, to be fair).I continue to be impressed by all the random awesome stuff we pulled off that we didn't even have to do. The room in which teams met GLaDOS during endgame was spectacular, and creepy in just the right way, and Allen and Joel and Roger went to a lot of effort to throw it together—and it didn't even have any puzzles in it! Not only did we get Peter to write a Mario-themed pastiche of the Mendelssohn wedding march (which, I mean, I'm sure he would have done anyway), we actually got members of our team to rehearse it and perform it at kickoff! The parody of "Still Alive" and closing credits! The achievements! That Bowser costume! And so forth: a lot of my teammates really went above and beyond the call of duty in doing stuff that, if we didn't have them, nobody would have even noticed—but having them made our Hunt a much more awesome experience.
(In retrospect, though, I'm not sure the congratulations e-mails we sent out whenever a team solved a supermeta were worth doing if we didn't have time to automate them. I kept being terrified that I would send a congratulations note to the wrong team, or for the wrong supermeta. I don't think I did, but it was one more source of anxiety than I needed.)
In a future post or two, I'll make comments on specific puzzles!
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
no subject
Date: 2011-01-25 05:30 am (UTC)