Wednesday, November 09, 2005

Crap Formats

One of the things I have enjoyed about my UPA job is that I have learned about formatting tournaments. I am certainly still in diapers compared to Adam Tarr, but I have enjoyed learning from him and the others in the formatting committee. I have great respect for their concern for fairness and well as their attention to detail. If you are a TD or tournament planner, I think the Formats Manual is a must read.

As I coach a college team at "social" tournaments, I am often presented with "napkin" formats. There are often serious problems with these formats. I suspect that a lot of the mistakes come with lack of knowledge or experience. However, when my girls asked one TD about a format, the response was "Its our tournament, we can do what we want". Not only does this acknowledge an awareness that problems exist, but it also signals that there was intent to take advantage of the organizational role. I am not going to guess as to what the intentions of a TD are, but if the intent is to gain an advantage, I do not understand how someone could put themselves in a position of upholding SOTG on the field, but clearly show disrespect in devising an unfair format. I have heard the "Hey, we are here to have fun and play some games...." rationale. It seems pretty clear to me that a team can have fun and "just play some games" equally well in a fair format.

I want to make clear that what follows next makes no assumptions about intent. It will point out some interesting wrinkles, but, honestly, my point is to say that players care about formats and is not trying point fingers or call someone a cheater.

So, exhibit A of crap formats is the tournament we played in this past weekend. The link is here:

http://www2.upa.org/scores/tourn.cgi?div=34&id=1943

The first thing that one notices is the decision to run it with 3 pools of 4. While the formats manual is very clear about the specific problems of doing this, I actually think that due to having only 4 fields, this is not a terrible idea.

The most significant problem is the way that the pools break into brackets. One notices that, for some reason, pools A and B graduate 3 teams while pool C gets to send only 2 teams. Again, I am not questioning intent, but it does raise a red flag when the the two host teams (Brown A and B) just happen to be seeded third in their pools. One would assume that the break between the C2 and C3 teams would be the 8-9 break. This would make sense as the top 8 teams make it to qtrs. Looking at the seeding, however, there seems to be no clear way to get to a place in which C2 and C3 can be seeded 8 and 9.

Looking at the brackets, the top game is A1 vs B3. So far so good. If one does a traditional snake (pool A: 1, 6, 7, 12: Pool B: 2, 5, 8, 11: Pool C: 3, 4, 9, 10), then this makes the qtr the 1 v 8 game. Let's look at the 2 seed, it is B1, down at the bottom of the bracket. Their opponent: C2!!. So not only does pool C send only 2 teams, but the reward...play the 2 seed. Again, if we are going with a traditional snake, this means we have the 2v4 game in qtrs. Seed number 3 (C1) is in the same bracket with the 1 seed, and they play A3, presumably the 7 seed. This leaves us with the 5 seed playing the 6 seed in the final game.

An additional problem with this bracket is that B1 and B2 are on the same side, meaning that, if one were to mis-seed and put the 2 best teams in pool B..... both could not make finals.

The other real problem is the seeding. I won't go into specifics, but Cornell at 5 is a joke, Yale above Wellesley (we beat them 12-3 two weeks before)doesn't work, and Brown-B at 7 is really an insult to everyone 8 and below.

So, I don't think it is particularly beneficial to be solely crtical, so I do have two alternate ideas. I think a strong possibility should have been 2 pools 6, stright into semis. 4 fields means playing the round robin in 8 rounds, so with semis and finals that means 10 rounds. A bit tricky, but possible. The other idea would be 3 pools of 4, but pre-qtrs on Sunday morning. This would have kept everyone in the ballgame, possibly corrected for seeding errors, and given 4 teams a bye for finishing in the top 4.

The teams that ended up getting screwed were BU and Yale. They finish 3 and 4 in C due to point diff, and then win the swill pool against, surprise!!!, Brown A and B.

In summary, if you are a TD, show respect for the teams that are taking time to travel to your tournament. Do the right thing and devise a format that is fair. If there is a need for deviating from standard fair practice, be very clear and upfront about why you are straying from the norm. Lastly, read the Formats Manual. It is a great resource.

19 comments:

parinella said...

That's nothing. In a tournament I attended recently, it was in one team's interest to let the other team score points!

Tarr said...

I was just getting some rest in the format-cave, and then I got the signal that this post was here.

Even with prequarters, the problems with 3 pools persist. Which 2nd place team gets a bye?

How about:

4 pools of 3(!)

A: 1, 8, 11
B: 2, 7, 12
C: 3, 6, 9
D: 4, 5, 10

Crossovers: A1-C1, B1-D1, A2-C3, C2-A3, B2-D3, D2-B3. Everyone gets 3 games, just like with 3 pools of 4. Takes the same number of games and rounds.

Sunday: Consult the 16/3 format and seed quarters accordingly. Then run the format they ran (5th place bracket, plus consolation pool for bottom 4).

gcooke said...

Jim,

You know me...I ain't happy unless I am whining.

Adam,

Excellent!

That makes a lot of sense.....didn't you teach those Brown kids anything when you were there?

parinella said...

I was just trying to hijack the thread to talk about the Nationals format. May I?

gcooke said...

Jim,

Seriously. I am touched that you would ask.

Please hijack away......

Sam TH said...

Although I had nothing to do with the tournament, it might be noted that it was only half of the Yale team that was beaten by Wellesley 12-3 (since they played with split squads at Coffee Cup). Not to take anything away from Wellesley's impressive success so far this fall.

This leads to another question: both Brandeis and Wellesley have improved very markedly in a short time, seemingly just by adding a coach. Is this something that was particular to those teams, general to previously low-level college women, or applicable to just about any team?

gcooke said...

Hey Sam,

If I had seeded that tournament, I would have still seeded us above Yale....just due to the win. When I seed Nats, for example, if the data says that team a played team b and won...well, they should be seeded higher. This is what is positive about a season, Yale comes out and beats us this past weekend. Now we have more data....

While I would love to take the credit that you kindly offer, Wellesley has had a bit of success in the past. They made the game to go in 2000 (lost to Jessi Witt and Smith). We also had a good year in 03 (my first year). We had 14 seniors, but lost to Tufts 10-9 in game to go to Sunday at Regionals.

I feel we are developing a better system which allows new players to get up to speed quickly, and the vets are showing more and more confidence. In terms of the big picture, I would love for us to develop the numbers to have a steady B team.

I make sure that the captains set the goals for the team for the year, and, in general, they are responsible for setting an example for what is expected. Now that I am in my 4th year, I feel I am finally getting the confidence to know what to say and how to say it.

I think an interesting trend will be to see how volunteer coaches react to a team going through a rebuilding year. For example, I understand that Ted M is not going to be coaching Brown this year. He has done a great job there, but there is so much turnover that that team essentially has to build itself from the ground up.

parinella said...

Hijacking...
Is it time to consider tweaking the Nationals format?
Last year, Pike played the same schedule as Sockeye the first two days, had a worse record, yet finished ahead of them in the power pool and avoided the pre-quarters.
This year, it was in Bnogo's best interests to allow Safari to stretch a five goal lead into a ten goal victory (whether they tanked on purpose, semi on purpose, or not at all is up to debate).
Should the format be changed? In the first case, it was never to anyone's benefit to get a worse point differential, but you still had the case that the higher finisher went into the next round carrying over an 0-1 record.
Option 1: give the higher finisher a 1-0 record to carry over, with a minimum +1 point differential. I had mistakently thought that Sockeye had finished first in that pool, but they missed by a single point. But it's still true for the Bnogo case, where they finished 4th and because of their blowout loss their vanquisher advanced to the power pool while the team they beat dropped down with a loss.
Option 2: change the power pool so that there is no carryover. Pool E would be A1-B2-C1-D2 instead of A1-A2-C1-C2. This would necessitate an extra power pool play game, which you'd have to figure out how to fit in. Tuneup used this format (in two days!), with the power pool fourth place finisher dropping down into the loser half while the B pool winners moved directly to quarters.

The bottom line is that it should never be in the team's interest to have a worse point differential. (It's almost unavoidable that a team might find it advantageous to not play in order to rest up, but it shouldn't help their positioning in the tournament. To this end, perhaps the first day format should mimic the change implemented for the second day this year, so that the round 2 game on day 1 will be between a first-round winner and a first-round loser, to avoid the Sockeye-Condors game where Sockeye could have lost 15-0 and finished first and the Condors last.)

gcooke said...

Unlike Zaz, I haven't worried too much about whether the views posted here represent the UPA. I will say that there is a review of the formats by the formats committee, and the Nats format is no exception. We did make some changes this year to address some issues raised last year.

It seems to me that there has been more "gamesmanship" of the past three years. Do folks agree?

Tarr said...

I'd guess a lot of the increased gamesmanship is simply due to increased parity. The inconsistent results are what create opportunities for gamesmanship.

On a related note... this year was an unprecedented 6-0 sweep for the play-in teams. Perhaps it is time to expand the play-in games, letting 2's from the lower pool play 3's from the upper pool? Or perhaps this means that the bottom pool path is relatively easy, and we need to move the play-in game to Saturday so that the top pool teams have time to get over their losses and prepare for prequarters. If you do that, there's room on the schedule for Jim's option 2.

There's also been talk on RSD about moving semifinals to Sunday, since the finalists often seem out of rhythm. I don't know if I buy this, but it would make finals more like every other game of the year, which is a good thing, I think. Especially in college finals, teams often play a different rotation than they do the rest of the year.

gcooke said...

I don't know how many times we said, during Nats, "Remember, Sockeye lost its first game, played in the pre-qtrs, and won the tournament."

I think we need to be careful that we don't always react to the previous year's problems, and then tweak based on certain scenarios. This is not a comment on Jim and Adam's tweaks, as they make a lot of sense, as much as a comment that I think gamesmanship is inherent in a format that is very non-committal in terms of eliminating teams.

I do think that the "sorting" period (Thurs-Fri) is positive to correct for seeding errors. Lord knows, I need the help with this.

I think a crucial decision that the tournament makes is that it divides the teams in half for qtrs. As Jim said in one of his comments, the season really begins with the elimination rounds on Sat.

What about, after a couple of days of sorting, the elimination rounds beginning with a round of 16? We acknowledge more parity. Most of our tweaks are centered around figuring how to do the pre-quarters in the best possible way. While pre-quaters are exciting, I think it is pretty clear that there are concerns as to how fair these are, based on what occurs earlier in the day.

I have always liked big brackets of more than 8 teams. They are inclusive, have the ptontial in a time of parity for great excitement, and the cornerstone of our formats is the 16-team double elim bracket format.

I have not thought at all about the specifics of scheduling. I am mostly addressing the philosophy and mechanics of dividing the teams in half for qtrs.

Flo said...

The big problem with going with a full round of 16 is that your best strategy with a short roster might be to not waste energy before that round---especially if parity is increasing you don't gain all that much by being a 1 as opposed to a 12-seed or so.
If a few teams think that way, the first two days are absolutely meaningless: no easier schedule if you win than if you loose. Might as well play a 2-day tourney straight elimination.
On top of that most strength wild cards will be decided in that round.

parinella said...

I think we need to start over with the basics of scheduling:
1. Teams should get a good number of games.
2. A fluke loss early should not eliminate a team from contention. (This is why the 3rd place team on day 1 is not eliminated from contention, because they could be 2-1. In the old 2 pool format, a team could be eliminated with a 5-1 record, yes, but in that case each of the teams had to beat an additional three teams to remain with one loss, whereas in 2-1 they might each just have beaten the worst team at the tournament.)
3. Every game (and point, for that matter) should mean something. I would go so far as to say that rule 0 for formatting is that a teams should not have a better placing as a result of losing a point or game. (It could be in a team's interest if they're short on players, or if they want to avoid matching up with a certain team (suppose half of NYNY 1991 missed their flight and the first game of Nationals and got sent to the B pool and were the likely pre-quarters winners, and you were playing off for first in the power pool; I don't think a format should say "the first place team gets to pick their opponent").
4. Seeding shouldn't matter that much, but we need to be aware that results aren't always consistent. I think some of the current formats are flawed because they assume consistency and terrible seeding and so result in absurdities.

gcooke said...

My comments on Jim and Flo's post.

I agree with Jim that a review of the basics is a good idea. I think this is what I was trying to say when I said that we shouldn't necessarily make changes to address all the possible wrinkles that occur.

I would say that of Jim's 4 points that probably 3 is the one that is under the most scruntity. Points 1,2, and 4 are addressed, for the most part, in my opinion.

In terms of Flo's comments:

"The big problem with going with a full round of 16 is that your best strategy with a short roster might be to not waste energy before that round---"

I am not sure that the format should be concerned with the size of a team's roster. Team's currently save energy in games that have little meaning. Also, I think we can see from Jim's comments that players do "coast" until Saturday in the current format.

"especially if parity is increasing you don't gain all that much by being a 1 as opposed to a 12-seed or so."

I guess the question becomes, is it worth it to expend the energy to try to determine the top 8 teams, even with the gamesmanship, by Sat morning?

"If a few teams think that way, the first two days are absolutely meaningless: no easier schedule if you win than if you loose. Might as well play a 2-day tourney straight elimination."

I think that we are starting to see that, other than going 0-3 on the first day, that the current importance of Thurs and Friday is diminishing.

"On top of that most strength wild cards will be decided in that round."

I guess I don't see why that is problematic.

There are times where I consider the format a bit wishy-washy. It does set out to "correct for seeding" on Thurs and Friday, but, to go back to Jim's point, does it really do this in best possible way? On the other hand, there are assumptions in the format (correct assumptions, in my opinion) that rely on the seeding to be "correct" in terms of a raanking of strength.

So the format is created based on an assumption that the seeding is correct, but then we say "oh, well, in this format...the seeding doesn't really matter".

I guess my point is this: in the presence of a real season and real data...it would be great to have a format in which seeding did matter. Or...if we are not confident about that...then devise the best way to get correct seedings before the elimination rounds (be it 8 or 16 teams) begin.

parinella said...

Responding as if it's my blog, this year was an unprecedented 6-0 sweep for the play-in teams
At least part of this is that the teams that got sent down on Day 1 may have actually been the "better" team (in Open, the 7 seed beat the 14 seed and 11 beat 13; in Women's, 4 beat 14 and 16 beat 13; in Mixed, 11 beat 7 and 10 beat 3 but who knows how the hell those Mixed teams are seeded?) But the reason this round exists is so that a team that goes 2-1 on Day 1 is not automatically eliminated. We were fine with eliminating a 1-2 team from contention, as that had always been the way it was.
I would say that of Jim's 4 points that probably 3 is the one that is under the most scruntity. Points 1,2, and 4 are addressed, for the most part, in my opinion.
Those points might be addressed currently, but if there is a change for point 3, the other points need to be revisited to look for unintended consequences.
coasting
I'm not sure everyone else has the same pragmatic view that I do. Twisted seemed to be playing their stars against us well past the time where they had any realistic chance at winning, even though they had the pre-quarters immediately following. It's probably only with injured or older players that most teams would consider resting them. gamesmanship
I helped come up with the format back in 1998, and we were definitely aware of the Pike Paradox, but thought that it would be rare (and it has been, occurring just that one time in 8 years), but probably weren't aware of the Bnogo Blow, or else thought it less significant since it was in the powerless pool. Definitely hadn't thought about the tweak that was made this year to pit 1-0 against 0-1 in the first round on Friday.
strength wild cards
You'd rather that they be determined over the course of the tournament rather than in one round, especially if teams don't care that much about where they finish in pool play.
consolation games, while we're at it
What if we used the earlier results instead? For instance, the quarters losers in Open were Bravo (finished 2nd in power pool), Subzero (3rd), Pike (played up from pre-quarters), and Doublewide (ditto). Why not award Bravo 5th, Subzero 6th, and let Pike and DW play off for 7th? And hadn't Hang Time and Slow White earned something with their 5-0 records prior to losing in the quarters?

gcooke said...

"strength wild cards
You'd rather that they be determined over the course of the tournament rather than in one round, especially if teams don't care that much about where they finish in pool play."

I really haven't thought this through. It seems to me that teams don't think about the strength wildcard until Sat morning. Then they are either in the running or not. The only time I have been aware of a team perhaps tanking a game due to the strength wildcard was in 04 when Shazam played RFBF in the last round of pool play. Shazam realized that if they lost, RFBF would be guaranteed a qtrs bid, and the NW would get a strength bid. They decided to play, won, and RFBF lost in pre-qtrs, but ended up winning the wildcard anyway. Perhaps if teams beacme aware that a round of 16 served as a "wildcard sorter" (those in contention and those not...it would give pool play more meaning.



"consolation games, while we're at it
What if we used the earlier results instead? For instance, the quarters losers in Open were Bravo (finished 2nd in power pool), Subzero (3rd), Pike (played up from pre-quarters), and Doublewide (ditto). Why not award Bravo 5th, Subzero 6th, and let Pike and DW play off for 7th? And hadn't Hang Time and Slow White earned something with their 5-0 records prior to losing in the quarters?"

I searched and searched and couldn't find the original quaote that Jim used for this reference. Is this a comment about playing consolation games?

Jim, I am curious about the process of creating this format. Can you elaborate a bit about the process and the various issues that you guys considered?

parinella said...

It was hashed out fairly quickly. The format was based on the April Fools format. However, Fools didn't have the pre-quarters game, and instead relegated the 3rd-place teams to the bottom half. We didn't want to let a 2-1 team be eliminated in a 3 way tie, so that meant that there had to be a play-in game. We also didn't want to add a round, since the existing format had 8 rounds over 4 days, so we didn't want to go to the Option 2 listed above, which is what Tuneup had used (along with relegating the 4th place team in the power pool to the 9-16 bracket. With fewer top teams back then, there were fewer big upsets and so a good placement in the power pool was more important.
We liked this format because it also meant that more of the games should have been against similar teams. In the 14 team 2 pool format, the #1 team would play 4, 5, 8, 9, 12, 13 in pool play and 2 and 3 in semis/finals. in the 16 team format, they would play 8, 9, 16, then 4, 5, then 7, 3, 2. That's a trade of 12 and 13 for 7 and 16, which means one more competitive game.

I'm not sure I've really added much here.

parinella said...

Another option would be to have a team carry over its entire record into the next round, not just the one game. This wuld come into play only if there was a three-way tie on Day 1 and might give better results. Instead of:
Furious (3-0)
Pike (1-2)
Bravo (1-2)
Sockeye (1-2)

it would have broken:
Furious (5-0)
Sockeye (3-2)
Bravo (3-2)
Pike (2-3)

And had Pike finished that day with the same record as Sockeye, they would have won a tiebreaker.

Bnogo still would have been better off throwing the game in order to get Homebrood in their pool, but they wouldn't have been in as good of a position, since they wouldn't have had a leg up on a potential three-way on Day 2.

Richard said...

Once I played a tournament in Northfield, MN hosted by St. Olaf. At the end of Saturday (which only had pool play), the TD announced that the open finals would be Sunday at 1 and would be between Carleton and Carleton-GOP. Thus, all Sunday games were moot. We showed up about 10 minutes before our game on Sunday, and found another team playing the team we were supposed to play, so the TD asked us, "Is there any team you'd like to play? I'll find you a game, don't worry". What a great tournament.