Actually, You could do the SUN thing and make it LGPL
Its what open office is under, and open office is also sold commercially as star office, with proprietary stuff added.
You would have the advantages of open source, but the but the option of a commercial venture,
even though you could make money with it being gpl, it would probably be support, or selling compiled games.
the alternative route would be go the ID software way and not opensource it until you cant make money with it anymore, so you just release the source to the community without any media, so people still have to make tiles and 3d modules, and add the parts of the engine that were removed when it was open sourced.