wolog.net

My brother is a rock star!

2008-06-20T02:38:30Z

He's all over the Health section of Google News today. Experimental Cancer Therapy Stops Man's Melanoma at WebMD; also see the BBC, ABC, Wall Street Journal, US News, and many others. He's been working on this line of immunotherapy research for many years.

I can't tell you how proud i am to be related to this guy. :)

Musical tuning: the reprise.

2008-06-03T01:21:50Z

I'm giving the musical tuning lecture again! Enough people missed it, and wanted to see it, that i decided to schedule a second run. Tomorrow night (Tuesday), Kingman Hall, 8:30pm in the fireplace room. Feel free to drop by.

Musical tuning: visuals from the lecture.

2008-05-24T06:37:17Z

The lecture went well! I had a good, engaged audience — a little over a dozen people, i think — and they put up with the multiple glitches in my presentation software. I wrote the software to do these animations using Pyglet, a multimedia library for Python.

Here's a condensed version of the first part of the lecture. It isn't quite the same without the audio — we could play the notes depicted on these slides, to hear the intervals we were talking about. But i'll try to explain the ideas in text.

* * *

Musical notes are oscillations, and the pitch of a musical note is determined by the frequency of the oscillation. (There are often many different frequencies embedded in the sound of a particular instrument, but the pitch we hear is usually the loudest and lowest of the frequencies.)

Two notes played together sound harmonious if their frequencies are in a ratio of small integers. That's because when the ratio is simple, the combination of waves makes a simple repeating pattern like the pair of waves on the bottom of this picture. The pair on top are not in a simple ratio, so the pattern doesn't repeat exactly.

The ratio between 300 Hz and 600 Hz is 1 to 2, the simplest possible ratio of two different notes. We hear that as an octave — such a basic interval that the two sound like the "same note". Here are the four octaves starting from 100 Hz — they go up to 200, 400, 800, and 1600 Hz. Plotted on a logarithmic scale, all ratios of 1 to 2 appear as equal distances. I'll mark this distance with a red bar.

Let's zoom in on one of these octaves and see what other ratios look like. I'll drop the "Hz" for now and mark the starting frequency just as 1. The next simplest ratio is from 2 to 3; i'll mark that with a blue bar. The ratio from 3 to 4 gets a green bar. And the ratio from 4 to 5 gets a yellow bar.

The ratio from 2 to 4 is also an octave. And the blue interval and green interval fit neatly inside — so they add up to exactly an octave. The blue interval is known as a "perfect fifth" and the green interval is called a "perfect fourth" — the reason for these names will become clear in a moment when we've put together the major scale.

If you go a perfect fifth up from 1, you get to 3/2, or 1.5. And if you go a perfect fourth up from 1, you get to 4/3 (about 1.333). These fit symmetrically between 1 and 2 — you can imagine a line right down the middle, and the two notes on one side are a mirror image of the two notes on the other.

The yellow interval — the ratio 5/4 — is called a "major third". If you go up this much from 1, you get to 1.25. And the three notes 1, 1.25, 1.5 make a nice-sounding chord — this is the major chord. Let's construct a major chord starting at each of the three notes shown above: starting from 1, from 1.333, and from 1.5. Each major chord is a yellow-blue pair in the picture below.

The top note of the chord starting at 1.5 overshoots the top of the octave — it goes to 2.25, which is the ratio 9/4. The equivalent note within the octave is exactly half of that, at 9/8. These seven notes we've identified form the major scale — if we label them in increasing order starting from C, these are (approximately) the white keys on a piano. So this explains why there are seven white keys in each octave on a piano.

You can see now why C–E is called a "third" — E is the third note counting from the left, and C–F is called a "fourth" — F is the fourth note, and C–G is called a "fifth" — G is the fifth note. The frequencies you see above are the "just intonation" for the major scale.

Notice that the spaces between the notes are uneven — E and F are closer together than the rest, and so are B and C at the top of the scale. In each of the five bigger gaps, there's space for another note. These are the black keys on a piano — that's why there are five black keys in each octave.

But piano keys are not tuned exactly like the notes in the picture above. That's because the spaces that look about the same are not exactly equal. For example, although D looks about halfway between C and E, it isn't. The ratio of D to C is 9/8 = 1.125, but the ratio of E to D is 10/9 or about 1.111. That means that if you tune your notes according to these ideal ratios, the scale will only sound right when you start on C. The other intervals will be off. For example, the fifth note starting from D is A. But the interval D–A is not a perfect fifth; it's 40/27, or about 1.48, which sounds really off.

To make it possible to play in any key, pianos (and most modern instruments) are tuned so that all twelve notes are equally spaced within the octave. This picture compares the justly-tuned major scale (on top) with 12 equal divisions of the octave (on the bottom).

By an amazing coincidence, when you divide the octave (a multiplicative factor of 2) into 12 equal parts (each a multiplicative factor of 2^1/12), you get notes that closely correspond to every note of the major scale. For example, 7 of these 12 parts almost exactly make a perfect fifth: 2^7/12 is very nearly equal to 1.5. Where the lines match up, you have a white key on the piano; and where there is a missing line on top, that's where you have a black key.

When you tune the piano to this system, which is called "12-tone equal temperament", the scale starting from any key sounds exactly the same. This means you can write music that shifts from key to key freely, without fear of ending up in a bad-sounding scale — all the scales sound equally good. The scales are a little off from what they should be, though. The fifth on an equal-tempered keyboard sounds pretty much indistinguishable from a real perfect fifth; but you see how the E doesn't quite match up? The major third is off by an amount that you can hear if you listen closely. The A is off by even more.

One of the big surprises for me when i learned about this stuff is that the 12-semitone system is rather arbitrary. It's not mathematically fundamental; it's a compromise: an approximation we invented. All of Western music is based on it, which, in a way, makes all of Western music a kludge. And it is impossible to tune the notes both in simple ratios and allowing modulation freely between all keys. The frequencies can never match up, because the just tunings are rational numbers, and any equal division of the octave will produce irrational numbers (nth roots of 2).

(I then wandered off into talking about the scales in other cultures, and we listened to samples of music from these other cultures and from modern experiments in 19-tone and 13-tone equal temperament.)

Here are all the slides, in the order i showed them.

Musical tuning: in the Kingman Lecture Series.

2008-05-22T21:13:08Z

I'm giving a lecture at Kingman tonight at 8 pm on musical tuning systems. Ever wondered:

...why there are 12 notes in an octave?
...why there are 7 notes in a major scale?
...why they have the pitches they have?
...what pitches the Ancient Greeks used?
...what notes make a scale in Arabic music?
...what music in a 13-note scale sounds like?

Come and find out.

Imitating greatness.

2008-05-18T03:10:10Z

Consider this a sincere form of flattery, then.

Conversations These Days

(The backstory: a friend of mine complained to me about how her dad doesn't answer her questions anymore; when she asks him something, he just tells her to look it up on Wikipedia. The trueness of this comment as applied to life in general made me think, this is the perfect kind of thing for an xkcd... so i decided to draw one.)

Thank you.

2008-05-05T05:46:53Z

If you have ever loved or cared for me, i would like to say thank you. I realized i wanted to say this a little while ago, when i was cleaning up my room and came across some of the letters and notes i've received from people who were close to me. They reminded me how incredibly lucky i've been.

You know who you are, and i'm grateful to you, even if we've fallen out of touch or out of favour with each other since then. Being cared for by you was and is a privilege. It has helped to make me the person i am. All i am is a collection of experiences shared and things learned from other people, and some of the best parts have come from you. Sometimes i forget this. I'm going to try harder to remember it, and to carry forward what you've given me in the way i live my life.

Zombies: the movie.

2008-04-26T04:17:12Z

This is one of the funniest and most awesome things i have read in a long time. Definitely worth checking out, especially if you ever wonder about the nature of consciousness.

GENERAL FRED: Are you sure?

SCIENTIST: As sure as we can be in the total absence of evidence.

Post-[Mukasey at the Commonwealth Club].

2008-03-27T23:21:03Z

When i arrived at the Intercontinental Hotel i was pleasantly surprised to run into tongodeon, who was there with a plan of his own. He brought the Attorney General a nice present — a Casio F91W wristwatch, with a polite and gracious note.

Indeed, my question was too long — much too long. I shortened it by half to write it onto an index card, which i brought with me to hand over. Then i had to shorten it drastically again to re-write it onto the special Official Commonwealth Club Question-Asking Card that they provided. (Yes, i misspelled Miers' name.)

Mukasey spent his entire speaking time lecturing us on government corruption. The hypocrisy was really quite breathtaking: he went on at some length about being a nation of laws and not of men, about the importance of providing a check and balance on every part of government, about ensuring that public officials are accountable and corruption be vigourously pursued and fairly prosecuted wherever it is found, regardless of political party. It was all i could do to keep myself from interjecting, "Yes — so stop protecting the President and hold him to the rule of law like everyone else!"

At the end of the session, the moderator did finally ask my question, or something on the topic, anyway. But she didn't read what i wrote; she filtered and edited every question down for politeness. The closest she got to anything resembling criticism was to say there were some questions from the audience about his consistency with respect to corruption. All she asked him was to offer his comments on the subpoenas on Bolten and Miers. So he simply repeated his old explanation: that the President's advisors should be able to talk privately to the President without being held to account. It was never pointed out that the law gives them no such permission, that neither he nor the President is allowed to tweak laws to suit their taste, or that he is protecting the Justice Department itself by ordering the Justice Department not to do its job.

He sounded so sincere and passionate in his main talk that i began to wonder if he simply has a blind spot when it comes to this executive. Perhaps he really has no idea that his actions are direct contributors to a dangerous consolidation of corruptible power in the White House. He continued to defend immunity for the telecom companies, the executive privilege to evade subpoenas, and his refusal to act against waterboarding without even the slightest awareness that there was anything wrong with letting the President freely order people to break the law.

All in all, it was probably not worth the money i paid for the ticket. But it was nice to see tongodeon again. An audio recording of the event should eventually appear online.

P. S. The Secret Service was there in force — at least a dozen men in dark suits with the trademark coil of clear plastic cord running from the left ear to the shirt collar. I always thought it was a Hollywood thing, but no, they really do get that far-off look and touch their ears with their hands when someone is talking to them on the radio.

P. P. S. (Added.) Thanks to tongodeon for providing this link to the prepared text of Mukasey's speech. Choice quotations include:

"We are, as we proclaim repeatedly, a nation of laws, not men."
"The survival and prosperity of a government of laws depends in great measure on the integrity of the men and women who pass, enforce, and administer the laws by which we are governed."
"When a public servant at any level of government exploits his or her office for improper purposes, the damage is measured not just in dollars and cents but also in erosion of the public trust – upon which depends the survival of our system of government."
"The investigation and prosecution of public corruption is therefore among the highest obligations of law enforcement."

Mukasey at the Commonwealth Club.

2008-03-27T09:38:11Z

I found out that Attorney General Michael Mukasey is speaking at the Commonwealth Club tomorrow. I decided to buy a ticket and attend, figuring i might not get a chance like this again. I hope to ask him:

Title 2, Section 192 of the U. S. Code says that refusing to testify or produce documents in response to being summoned by Congress is a misdemeanor punishable by fine and imprisonment. It does not equivocate. It doesn't say it's a misdemeanor unless the President prefers someone not to testify. Section 194 says that when someone violates Section 192, a U. S. attorney has the duty to bring the matter before a grand jury. It doesn't say that this duty only exists unless the President prefers the law not to be enforced.

I refer, of course, to the violation by Mister Bolten and Miss Myers of Section 192, and the refusal by Mister Taylor to perform his duties under Section 194. Surely you are aware that there is no hope of upholding the rule of law if those who are duty-bound to enforce laws choose to enforce them selectively, at the whim of those against whom they would be enforced. Will you now commit to perform your duty as attorney general and uphold these laws, or will you explain to us, assembled here in this room, why you and Mister Taylor should not be prosecuted for obstruction of justice, and why anyone should believe that you are serious about eliminating corruption when it is your own department that is protected from scrutiny by your continuing failure to enforce the law?

If you know anything about how these sessions are run, or how one gets to ask questions, please do tell.

Factville: a new project.

2008-03-25T11:49:44Z

I'm starting a new open source project. It's something i've been thinking about for quite a while now, and have mentioned to people here and there. Let me tell you a bit about it.

Exhibit A: a book on gender differences

A little while ago, i wrote about a newspaper article in the SF Chronicle. The subject of the article was a new book by Louann Brizendine called The Female Brain. On the cover of the book is a brain-shaped mass of white plastic telephone cord, the old kind that comes in a long springy coil — a visual wisecrack depicting the book's central claim that women are born communicators ("excess testosterone shrinks the communications center").

The book jacket lists several gender stereotypes as bullet points. One of them is a specific numerical claim: A woman uses about 20,000 words per day while a man uses about 7,000. Other sources give a wide range of numbers, from "7,000 vs. 2,000" to "50,000 vs. 25,000". Perhaps a lot of people believe women are inherently more talkative. But there doesn't seem to be much evidence to back this up. Actually, a recent study suggests that men and women talk about equally much.

Nonetheless, Brizendine's claim was quoted all over the media. It made a huge impact (the book was a bestseller), and a considerable amount of time went by before it was debunked. To a casual observer, the claim probably doesn't even appear to be debunked at all: a reputable scientist says one thing, a little while later another scientist says the opposite — who's to say which is right? Another virtual throwing up of the hands, another shaking of heads, another anecdote about those silly academics who can never agree on anything.

Catching and recovering from misconceptions

Of course, this sort of thing goes on all the time. Brizendine, as i said, is a reputable scientist — she is a medical doctor and has been on the faculty at Harvard and UCSF. Plenty of facts and figures quoted in the media are presented by people who don't even claim to be scientists or to have evidence. Public misconceptions are pervasive, stubborn, and can be enormously costly.

When you come across a fact — or something that's claimed to be a fact — how do you know whether it's true? Maybe you Google for it; after all, the Web is somewhat more democratic than the TV and print media. But the Internet is also notoriously good at spreading rumours. Maybe you check Wikipedia, trusting its community editing process to do a good job of weeding out errors. Or perhaps you visit Snopes, hoping that the rumour you heard is common enough that someone there will have written an article about it, and you think the people who run that site are pretty decent at what they do.

On the other hand, Wikipedia and Google are a little too general: they may give you an article that's generally related to your topic, and then you need to examine it to see if it mentions the particular claim you want to check. And in both cases, the filtering process is hard to examine: Google's ranking algorithm is secret, and although at Wikipedia everything is public, you could spend weeks reading through the discussion pages trying to find out how a particular claim got inserted into the article. Snopes offers an excellent overview of each rumour, but there's only so much that two people can write. And of course you have to trust those two people.

An idea for a new service

So i think there's a useful service that could be provided by a new website: something with the openness and democratic participation of Wikipedia, but more focused on specific claims and the evidence for them. Thus Factville: a community-edited database of facts and supporting evidence. The site i have in mind would not be an alternative to Wikipedia, but rather a tool to help Wikipedians. A large part of the debating on Wikipedia consists of people gathering sources to support statements they want to put in the article; Factville could help them organize these sources and settle these debates. Factville would also be a tool for bloggers and journalists. When a controversial claim appears in the media, articles spring up all over, taking sides on the claim, quoting and citing sources to support their position. Why not have a place to gather the complete list of sources? Why not discuss them and rate them, the way the Web has taught us to discuss and rate photos, discuss and rate URLs, discuss and rate movies?

That's what Factville is about. It's going to be a Frankensteinian cross between Wiki-style websites (community-edited, completely freeform text, with a recorded history of changes to establish accountability) and Flickr-style websites (community-maintained, structured information, with tags, comments, and ratings). The big challenge will be to make this simple and easy to use. Here is the basic design:

The site is a database of claims.
Each claim has lists of supporting and refuting citations.
Each citation quotes from a source and explains how it relates to the claim.
Each claim can also have supporting and refuting arguments.

Each of these things is an editable page, with associated discussion and rating tools.

A source can be any kind of published work — a newspaper article, a conference paper, a video clip, a blog entry, etc. Some sources stand on their own (like Brizendine's book); others belong to a publication venue and rest partly on the venue's reputation (the credibility of an article in the New York Times is related to your opinion of its editing standards).

Citations and sources are separate things because the same source could be used for several claims, or even cited as evidence on both sides of the same claim (perhaps quotations excerpted from different parts of the same source). Information on sources could also be automatically drawn from the syndication feeds of popular publications.

When a contributor wants to put together several sources or other claims on Factville, and combine them into a reasoned case for or against a claim, they can write an argument. Other visitors can rate the arguments up or down so that the most convincing arguments get the most attention.

The ratings of claims, citations, and arguments are not supposed to tell you what is true. They can only tell you about other people's opinions. But the goal is to give you as complete as possible a view of all the evidence, and to let the collaborative power of a large crowd help you find the most relevant factors to consider, as you make your own decision whether to believe each claim.

A modest start

I don't have a running website yet. I have a lot of ideas, some in my head and some written down, many in this journal entry. And i have a start at some code that implements the database structure i just described. Today i registered a new project a Launchpad, an open source project hosting service. You can monitor my progress on the Factville page there. The code I've written so far is available from that page. It's written in Python and runs on Django, which i'm still learning.

PyCon 2008.

2008-03-25T09:34:07Z

Last week i went to the Python Conference in Chicago. It was good to be among that crowd of familiar faces again. I've been feeling somewhat anxious and lonely lately in anticipation of the shift from school to Real Work and my imminent move out of the co-ops (yes, that's still months away but it makes me sad), so it's nice to be reminded of a community i can continue to be a part of. Quite a few Python hackers are at Google (including Guido himself), so i may be seeing more of them soon.

As usual, the hallway sessions were the best. Some of the talks were pretty fun too, but the best part of it for me was just feeling like part of the flow and catching up with all the interesting things people are working on. It's nice to be somewhere where most people either know who you are or are excited to get to know you.

PyCon was huge this year — something like 1100 people, compared to 600 last year. It was estimated that, assuming the current geometric growth rate continues, the entire population of the planet would be attending PyCon in some small number of years.

This was the first time i stayed afterwards for the coding sprints. Sprinting is a simple concept: get in a room with a bunch of other programmers and code all day. Even though it sounds just like work, somehow it's a lot more fun. I think it's a combination of working directly with awesome people (e.g. having the creator of Python sitting just two chairs over, or a co-creator of Django sitting across the table, so you can ask questions when you get stuck) and the focus brought on by the limited time and the specificity of the tasks. The atmosphere is different when everyone has flown in to be at the sprint and you know you're only going to be there for a few days. Travelling also gives you an excuse to ignore some of the sources of distraction that exist at home and at the office.

I did a little work on Python 3.0. It felt good to be productive and efficient. I got in 12 hours of work in a day, felt fine, and got decent, regular sleep. (I was hoping to keep it up, but alas, my sleep schedule immediately disintegrated and productivity plummeted upon returning home.) I've been carrying some guilt because i've been a long-time member of the core Python development team, with commit privileges and all, and have contributed absolutely nothing for the past few years. I think i get consistently too much credit in the Python community, both because of that unearned status and because of a bug-tracking program called Roundup that is awesome but that i didn't write. (Once upon a time, i wrote a proposed design for a bug-tracking program — basically a wishlist of things i thought would be cool to have. Richard Jones then did all the hard work to actually build it, but people keep complimenting me on it just because the name was initially associated with me.) The work I did at the sprint (which had to do with the handling of Unicode in the new file I/O module, if you're curious) was pretty small, but at least it was something. I hope i continue to have time to contribute.

What's next for me.

2008-03-12T20:23:33Z

I did finally make a decision about what to do next with my life. I'm going to be joining Google.org in San Francisco. The work they're doing is very exciting — they're taking on some of the really big problems in our world, and i'm thrilled to be joining them. I think two of the biggest factors for me are (a) the huge learning opportunity of working with experts in international health and development, and (b) the openness of the "dot-org" side of the company. Google's culture of intense secrecy was one of the main concerns i had about working for them, and this seems to be much less of an issue at Google.org.

As i understand it, Google.org is not a separate non-profit organization. It's part of Google, and in many ways i will be just another Google engineer. But it does have a distinct mission, and compared to Google as a whole it is relatively young.

I owe my thanks to everyone who had comments and advice to offer as i was pondering this big decision. Having the support of so many wise friends helped a lot. I'm also grateful to many friends who are now at Google, for encouraging me to look into this possibility and for whatever good words they may have put in on my behalf to make this opportunity real.

I can't wait to start playing my small part in efforts to solve some of our biggest problems!

Another world diagram, and an invitation.

2008-03-03T06:50:15Z

Here's my picture now:

Forces for good:

I realize that everyone probably has different ideas about what should go on this picture, and what arrows should connect them. I think that's interesting — what you choose to put on the diagram says something about your worldview.

So: take a few minutes to draw your own diagram. Then take a picture of it and post it in a comment here, or post it on your blog/journal and leave a comment here.

I'm curious to see what will show up in everyone else's drawings.

How long does it take to write a dissertation?

2008-02-12T10:03:18Z

This is the entry in which i confess to one of my compulsive behaviours. My sleep cycle is not the most regular thing in the world (but then again, i hear this is not so unusual for hacker types). Once upon a time i got interested in my sleep patterns,¹ and decided to start recording them. So i hacked up a little utility to help me keep a log of when i sleep and wake up, with fairly minimal effort.

When i started writing my dissertation, it occurred to me that it would be nice to know how much time i spent on the thing when i was finally done with it. So i made a habit of recording when i worked on that too. Now i have hundreds of little timestamped log entries.

What does one do with all this data? Visualize it, of course...

Time spent on the dissertation (orange): 327.9 hours
Time spent on the talk (purple): 41.4 hours
Total: 369.3 hours

(Made with Python and ReportLab, then marked up with OmniGraffle.)

Note 1. I also tried living on a 28-hour day (with a 6-day week) for a few weeks one summer. It was interesting. But that is a story for another time.

Future plans.

2008-02-08T15:47:28Z

So... i'm trying to organize my thoughts on what to do with my life. I've been out of school for a month, interviewing for jobs and coming up with project ideas. I decided that writing might help me sort this out, so i'm going to do some of my thinking here, in the open. I invite your reactions and opinions — some of you know me very well, and your feedback could help me out.

Desiderata

First, a little bit about what i'm looking for.

I am convinced that a capable computer programmer can build things of great benefit to the world. I'm not trying to be arrogant; i just think it's true because software and networking enable inventions to spread at incredible speed, and a single person can launch one with nothing more than a laptop and an Internet connection. The software industry is unique in this respect. It only took one person to invent HTML and two people to start Google or Wikipedia. Individual programmers have created things as powerful as Napster (at age 19), Facebook (at age 20?), and BitTorrent (at age 26). So, from a certain perspective, i'm already way behind the game in terms of fulfilling potential.

That's my primary goal: for my existence to have yielded things of benefit to the world — hopefully, of significant benefit to many people. That means my decisions hinge on a calculus of benefit, which is of course a complicated and subjective thing. I often find myself feeling like the stonecutter in the fable as i chase down chains of logic trying to figure out how to achieve maximum benefit. In any case, my current line of thinking is that there are five factors in choosing the most beneficial option:

number of people who benefit
degree of impact of the project
degree of my personal impact on the project
likelihood of success
necessity of my participation

The first three factors are straightforward: the bigger the better (i.e. greater contribution and greater fulfillment). But the fourth, "likelihood of success", is a tricky one: if it's too low, i am likely to be wasting my time on something that won't benefit anyone. If it's too high, then it interacts with the last factor: if a project is already certain to succeed, with or without my help, then contributing my help adds nothing.

The last factor implies that there has to be something about my skills that fits the project — if the job i do is something that would have been done by someone else anyway, then my choice to join the project has little effect. And the ultimate choice, in terms of the last factor, would be to start and launch something of my own, provided it doesn't duplicate something that already exists.

I think of these five factors as combining in a roughly multiplicative way — a × b × c × d × e is the approximate expected utility of making a particular career choice. (Let me know if you notice factors i've forgotten.) Notwithstanding all that, i am biased toward projects that benefit a large number of people and/or people who are less fortunate. I don't know to what extent this is because they are truly more useful, or because i want to be famous or seen as noble. But whatever the reason, it matters to me to do work whose benefit most people can understand.

What is the most important problem?

There's a saying about how to win a game of Go: simply always make the biggest move. Each stone you play will affect the final score somehow; if you choose moves that are worth more than your opponent's moves, you're bound to win. The hard part is evaluating what each move is worth.

I don't expect to save the world by myself, but i'll get further if i have the conviction to focus on something rather than dabbling in a lot of different projects. So, i feel it's time for me to pick a big problem to attack, and after i've chosen it, to go as far down that road as possible. The question is what problem to choose.

Below are some possible answers, presented as arguments by imaginary people (members of the committee in my head, you might say). I've also broken these out into top-level comments by me below so you can comment on them individually.

1. Energy crisis

All the things that support our modern society and well-being — production, transportation, manufacturing, health, communication, defense — run on energy extracted from our environment. Literally everything either uses energy or is produced by a process that consumes energy. Our energy extraction and consumption is starting to run into limits. Without an energy source that is efficient, affordable, doesn't start wars, and doesn't cause significant climate change, all of humanity is in deep trouble.

Therefore, the most valuable thing for you to do, Ping, is to devote your technical and engineering skills to developing renewable sources of energy or ways to conserve energy. It doesn't matter that this is a new area for you; you can learn, and we need all the help we can get on this problem. The potential damage caused by global climate change is so enormous that if we fail, nothing else will matter.

2. Computer security

Energy is an important problem, but it doesn't match your skill set. Other people are working on that problem. But all the same things that are listed in #1 as essential parts of our modern society and well-being also heavily depend on computer software. Computers are in control of ever more powerful things both virtual (governments, banks, financial transactions, personal information) and real (factories, weapons, critical infrastructure). Yet, even as we become more dependent on them, they are becoming less and less reliable. They break down often; they are hard to use; they behave and misbehave in unpredictable ways. And they are vulnerable to attack. As we rely on them more, small bugs, vulnerabilities, and human mistakes will have more and more catastrophic effects.

The computer science community needs to collectively bring about a change in the way we design and build software, and you're in a position to play a significant part in this effort. A combined approach to usability and security is essential to solving this problem, and you've brought them together in your previous research. Capability-based security thinking is also essential; you understand the capability ideas, and you're well-connected with the main players in that area. They are strong, but few. Join them. You already have momentum in this direction; use it to help them push the software world onto the right path.

3. Poverty and education

The real problem is peace. Without peace, we have nothing. Conflict comes about because of suffering, misunderstanding, and disparities between groups of people. The most effective way to bring a lasting end to poverty is education. To create understanding between different cultures requires education. To develop higher standards of living, address health problems, and start successful businesses requires education. To help people understand democracy and human rights requires education. To spread awareness of societies that are freer, fairer, and more humane requires education. The most powerful thing you can do is to help bring educational tools and global communication to those that don't have them.

The problem is urgent. Time is short. Increasingly destructive weapons are available to increasingly many people. International conflict is becoming more and more dangerous. The tools of peace and the tools of war are in a race, and you must help accelerate the tools of peace if peace is to prevail.

4. Governance and democracy

The real power is in the hands of governments. Peace is not achievable when politicians in powerful countries can deceive their citizens and launch wars with impunity. Solving the energy crisis, improving public health, safeguarding the environment, and the other big challenges are all dependent on politicians making good decisions.

But in the most powerful country of all, the United States, democracy is broken. Time after time, the political system arrives at the wrong answers to even the most straightforward and well-understood questions — whether retroactive copyright extension promotes the creation of new works (it doesn't), whether climate change is real (it is), how much it will cost to invade Iraq ($500 billion and counting) — despite objective analysis and a clear consensus. That's because the system is corrupt: the decision-making process is distorted by money and lobbyists, whose influence is so strong that it can obliterate facts.

The only way to restore democracy is to hold politicians accountable to the people who vote for them, instead of the lobbyists who appeal to their financial self-interest. Part of the problem is to fix the voting machinery and the method of counting votes to determine the winner; you've already done some thinking along those lines. The other part of the problem is to make the votes actually mean something — in order for votes to actually carry some power, government needs to be more transparent and accessible. Citizens need ways to understand what their elected officials are doing, hold them responsible for keeping their promises, and evaluate their campaign rhetoric in light of their actions. The UK has tools like theyworkforyou.com and hearfromyourmp.com; the United States and other countries need tools like these as well, and more. You can make this happen. Now is your chance to apply your web development, design, and information visualization skills, and use the power of the Internet to help fix democracy.

5. Media and journalism

Holding politicians accountable to their voters doesn't do any good if the voters are wrong. The idea that democracy will produce the right answers — faith in the wisdom of crowds — ignores the susceptibility of the populace to media influence. The media are biased to serve their own interests (for example, they aren't exactly eager to discuss media regulation, and prefer to emphasize the sensational over the significant) and are beholden to advertisers as well. Politicians aren't the only ones that put out misleading advertisements; corporate marketing departments regularly distort the truth too.

The conflict of interest inherent in commercial media makes it difficult for most citizens to get getting accurate, unbiased information on which to base not only their votes but also their purchasing decisions. The only way to solve this problem is to hold the media — and all sources of information — accountable for getting it right. The alternative of replacing the centralized media with a more distributed system like Wikipedia or OhMyNews is also appealing, but even if that were to happen, we would still need a way to hold sources accountable for their accuracy.

You made a start at tackling this problem before, when you tried to enable public annotation of web pages; there is plenty more that can be done to help the public substantively critique and compare news articles, blog posts, and primary sources. There is an unmet need here for a way to settle on facts backed by evidence and consensus — something like a Snopes for the media, with a participatory and scientific process. You've been wanting to create something like this for a long time; now you can make it real.

6. Decision-making and collective intelligence

The most urgent problem is to improve our ability to solve complex and urgent problems. Finding solutions is about much more than facts. Questions like how to address global climate change, whether nuclear power is safe, how to design secure computer software, and how to improve public health are complex and intricate. Right now, finding and evaluating solutions to these problems is a tremendously inefficient process of researching a mountain of literature, with a high likelihood of replaying misunderstandings and logical errors that have been made many times. This inefficiency also leads to fractured communities of experts with each one holding a narrow point of view, reinforced by the small segment of the literature with which they are familiar. The Internet has made access to literature faster, but has made the literature itself no more accurate or easier to integrate.

To be able to deal with the increasingly complex and urgent problems we face, communities of scientists and experts need more effective ways to lay out their evidence and their arguments, evaluate them, and rapidly arrive at a consensus. The status quo is decades of controversy and debate, sometimes with failed public policy experiments carried out at devastating cost. It is doubtful that we can afford to keep doing things this way. You've been interested in collective intelligence for a long time, and have dreamed of better ways to organize discussion and argumentation online, and to help sound reasoning win out over logical fallacies. Carry out that dream.

Your opinions here...

Which answer sounds the most compelling to you? Are there other good options i've failed to identify? I'm interested in your thoughts.

Update: I've already received several suggestions of the form, "Do what you enjoy." It's good advice, yes, but i should probably explain why i've intentionally left that out of this particular analysis:

For me, enjoyment and motivation go hand in hand. If i realize that what i'm doing isn't that significant after all, the motivation will go away and i'll stop enjoying it. So part of the point of this exercise is to construct an argument strong enough for whatever choice i make, such that i can maintain conviction in my choice and continue to enjoy it, long enough to actually achieve something.
Also, there's something that feels intellectually dishonest about choosing an answer just because i like it rather than because it's right. I'm hoping to find more solid grounding for my decision.

Why this primary matters.

2008-02-05T08:54:57Z

Larry Lessig explains the significance of tomorrow's primary.

I would go further; i think there are real policy differences between Obama and Clinton that matter, and that weigh decisively in favour of Obama (technology policy, for example). But i can't vote in this election, so i'll just share Lessig's video with you. Now is not the time to be afraid. Now is the time to reach for new hope.

zestyping ... you're an idiot.

2008-01-25T23:12:52Z

There are so many more thoughtful and significant things i've been wanting to post about for a few weeks now. I hope i get to them. But today i couldn't resist posting a link for you:

Best comment EVAR.

zestyping ... you're an idiot. I rest my case.
Why?

How about this comparison ... you can stick it in any language you want even your asss...
try this piece of code on any compiler....

if ("zestyping" > "all idiots around the world") {
echo "It's true, on all compilers";
}
...wow...I'm amazed ... it returns true, they should stop you from touching the keyboard.
OMFG.

I'm still laughing.

What does Iowa turnout mean?

2008-01-04T22:53:59Z

So by now everyone knows that Obama and Huckabee won the Iowa caucuses. But perhaps less well known is the dramatic difference in voter turnout — between Democratic and Republican, and in 2008 compared to previous years.

Iowa Caucus Turnout, 2000–2008

	2000

60 760

87 666

	2004

124 000

uncontested (George W. Bush)

	2008

221 000

116 114

Some of the numbers above are approximate, but the trend is clear: a huge increase in Democratic caucus participants (78% more than in 2004, and 90% more Democrats than Republicans this year). My understanding is that the Democratic caucuses also require more time and involvement than the Republican ones. All of this adds up to a lot of enthusiastic Democrats — at least in Iowa.

Here's what that looks like if you break down each bar according to the proportion of support that each candidate received.

Iowa Caucus Vote Distribution, 2000–2008

	2000

Bradl

Gore

Bush

Forbes

	2004

Dean

Edwards

Kerry

uncontested (George W. Bush)

	2008

Clinton

Edwards

Obama

Huckabee

Romney

Tho

What do you think this means for the election as a whole? Is this phenomenon local to Iowa, or is it a sign of things to come?

Sources: MSNBC, AP, CNN, Eric Appleman.

Contributing to Wikipedia.

2008-01-04T20:43:43Z

The Wikipedia article on Canada now has a spoken audio version.

The dissertation.

2007-12-22T05:46:22Z

If you're interested, my dissertation is available on my website and also in the Berkeley EECS Technical Reports archive.

Here is the abstract:

I examine the question of how to design election-related software, with particular attention to the threat of insider attacks, and propose the goal of simplifying the software in electronic voting machines. I apply a technique called prerendering to reduce the security-critical, voting-specific software by a factor of 10 to 100 while supporting similar or better usability and accessibility, compared to today's voting machines. Smaller and simpler software generally contributes to easier verification and higher confidence.

I demonstrate and validate the prerendering approach by presenting Pvote, a vote-entry program that allows a high degree of freedom in the design of the user interface and supports synchronized audio and video, touchscreen input, and input devices for people with disabilities. Despite all its capabilities, Pvote is just 460 lines of Python code; thus, it directly addresses the conflict between flexibility and reliability that underlies much of the current controversy over electronic voting. A security review of Pvote found no bugs in the Pvote code and yielded lessons on the practice of adversarial code review. The analysis and design methods I used, including the prerendering technique, are also applicable to other high-assurance software.

It turned out bigger than i expected — 324 pages — but don't let that scare you. That includes 14 pages of front matter, 10 chapter cover pages, 11 pages of bibliography, 102 pages of appendices (31 of which are source code), and 5 pages of the GNU FDL. In short, 182 pages of content and 142 pages of fluff. The content is broken up into little sections in an attempt to make it easier to read.

Many people contributed to the work. The more i learned about things that other graduate students have had to deal with, the more i realized how lucky i was to have Dave Wagner and Marti Hearst as advisors — they got back to me quickly, read drafts carefully, and had lots of well-thought-out and constructive comments to offer. Candy Lopez showed me around the election office in Contra Costa County and patiently explained to me how everything was done in real life. Noel Runyan and Scott Luebking taught me about accessibility, and i appreciate their advice very much even though the dissertation doesn't address accessibility as much as it could; the research didn't include user testing with disabled voters. Matt Bishop, Ian Goldberg, Yoshi Kohno, Mark Miller, Dan Sandler, and Dan Wallach volunteered a huge amount of time to review my source code. Joe Hall has been a great help on questions about election law and policy.

The type is Lucida; i chose it because it has a harmonized family of serif, sans-serif, and fixed-width typefaces. Alas, the original Lucida fonts are unkerned (and the word "Voting" shows up a lot, which makes the lack of kerning more obvious), so i added some kerning pairs by hand. The page design is inspired by Scott Kim's thesis on Viewpoint — he also used Lucida and a wide left margin for small figures and illustrations. It looks like his left margin was chosen to match the first of three equal columns, which would occasionally be used for three equal-sized figures. I moved the left margin so that the vertical edge divides the page in the golden ratio. Just because.

If you're curious about the topic of electronic voting, please check it out. I hope you find it interesting.

Update: Pvote has a website where you can get the source code and documentation, as well as a sample ballot definition file. Also on that site is the assurance document, which is the specification that was used for the Pvote security review, and a report on the results of the security review.

Wexler wants hearings on Cheney.

2007-12-20T10:31:11Z

Have you seen this? Congressman Wexler is pushing for impeachment hearings for Cheney to investigate his involvement in manipulating intelligence, authorizing torture, ordering illegal wiretapping, obstructing justice, and abusing executive power. To indicate your support, go to wexlerwantshearings.com and join the over 100 000 others who have signed up in just five days. Pass it on!

PhileD!

2007-12-20T01:36:17Z

I handed in my dissertation today at one minute before closing -- the day before the last day to file. Now i'm really done. What next? How about some sleepzzz....

When <BLINK> was hip.

2007-12-14T05:07:47Z

This has to be the most ridiculous conference website i've ever seen.

Why PHP should never be taught, Part II.

2007-12-12T02:35:12Z

This is a follow-up to Why PHP should never be taught.

% cat comparison.php
<?php

$a = -1;
$b = 0;
$c = 1;
$d = True;

print "d > $a: "; print ($d > $a) ? "yes\n" : "no\n";
print "d > $b: "; print ($d > $b) ? "yes\n" : "no\n";
print "d > $c: "; print ($d > $c) ? "yes\n" : "no\n";
print "\n";
print "d < $a: "; print ($d < $a) ? "yes\n" : "no\n";
print "d < $b: "; print ($d < $b) ? "yes\n" : "no\n";
print "d < $c: "; print ($d < $c) ? "yes\n" : "no\n";
print "\n";
print "e > $a: "; print ($e > $a) ? "yes\n" : "no\n";
print "e > $b: "; print ($e > $b) ? "yes\n" : "no\n";
print "e > $c: "; print ($e > $c) ? "yes\n" : "no\n";
print "\n";
print "e < $a: "; print ($e < $a) ? "yes\n" : "no\n";
print "e < $b: "; print ($e < $b) ? "yes\n" : "no\n";
print "e < $c: "; print ($e < $c) ? "yes\n" : "no\n";
?>

% php comparison.php 
d > -1: no
d > 0: yes
d > 1: no

d < -1: no
d < 0: no
d < 1: no

e > -1: no
e > 0: no
e > 1: no

e < -1: yes
e < 0: no
e < 1: yes
%

Aaaarghhh!!! It's so arbitrary.