Posts in category 'hacking'
Using Git as a Bridge to SVN
So, Lenore complained that I haven’t been blogging much lately, so I thought I’d throw together a little something that would bore her to tears. :) Now, as quick prelim, if you don’t know what Git or Subversion is, I’d just skip this one.
Now, this whole thing is really more of a note-to-self, as I’m still getting used to git. But I figured it might be useful to other developers. Now I should say this isn’t going to be a generic discussion about how to bridge Git to SVN. That’s be done to death all over the web. What I’m particularly interested in is how to use git-svn when your Subversion repository has an… unusual topology. See, the typical Subversion repository is laid out something like this:
projectA trunk branches projectA-1.0 tags projectB trunk branches projectB-1.0 tags
In a case like this, importing the tree into git-svn is trivial. However, at work, we have a Subversion repository that looks something like this:
trunk projectA projectB branches projectA-1.0 projectB-1.0
This layout requires some hacking to get git-svn working. First off, you need to initialize a standard git repository:
mkdir git cd git git init
Note, you’ll probably also want to use “git config” to set a few parameters while you’re at it (if you’re on Windows, setting “filemode” and “autocrlf” to false is a very good idea). Once the basic repo is set up, you can now add the SVN repositories you plan to fetch. For example, to pull projectA and projectA-1.0, we’d run these commands
git config --add svn-remote.projectA.url svn+ssh://path/to/svn/ git config --add svn-remote.projectA.fetch trunk/projectA:refs/remotes/projectA git config --add svn-remote.projectA-1.0.url svn+ssh://path/to/svn/ git config --add svn-remote.projectA-1.0.fetch branches/projectA-1.0:refs/remotes/projectA-1.0
Basically, we instruct git-svn what the mappings are between various tree names and their SVN equivalents. This includes a URL to the repository, along with a definition of where to fetch the code from. Once this is done, you need to populate your repository. Note, I think the first fetch becomes the master branch (I’m not actually sure about this, but it seems that way), so it’s best to yank trunk first (you can move the master moniker around, but why bother with such machinations if you can do it right the first time?):
git svn fetch projectA git svn fetch projectA-1.0
Note, somehow, and I have no idea how, git even manages to figure out the 1.0 branch parent, so the essential structure is preserved, even though we perform separate fetches to populate the trees.
So now we have the trees downloaded, and as a bonus, the master tree is set up and tracking trunk. Next, we need to create local branches tracking any remote branches we’re interested in:
git checkout projectA-1.0 # Switch into the remote branch git checkout --track -b 1.0 # Create a local branch named 1.0 tracking the projectA-1.0 remote branch
There, the tree is populated. Finally, if you have svn:ignore properties set up, here’s how you mirror them locally (note, you’ll have to do this for each branch). Now, in git, the ignore rules are stored in .gitignore files. This is fine, except that, by default, git wants to include those files in the git repository, and thus they’d get pushed upstream in a dcommit. Thus, we have to go through a bit of gymnastics to make sure that doesn’t happen, and that the .gitignore files remain local.
First, open up .git/info/exclude and add these lines:
.gitignore .gitmodules
This instructs git to ignore the .gitignore files (and .gitmodule files) on “git status” calls and so forth. Next, create the ignore files.
git checkout master git svn create-ignore
Of course, when this happens, git-svn goes and does a “git add” for all the .gitignore files. This is exactly what we don’t want. :) So, last but not least, we undo the adds:
git reset HEAD
And your tree should now be ready for hacking. Now for a few basic recipes. First, to switch between branches:
git checkout 1.0 # Switch to our projectA-1.0 tracking branch git checkout master # Switch back to master
Second, here’s how we update the current local branch to sync up with it’s corresponding remote SVN branch:
git svn rebase
Third, to commit a change to the local git branch:
git commit -a
Next, to push the changes in the local git branch out to the Subversion repository:
git svn dcommit
Note, a dcommit pushes each individual local commit to the Subversion tree as individual SVN commits. However, there may be times when you want to roll together a series of local changes into a single SVN checkin (perhaps your code went through a lot of churn before the final version was reached). Luckily, git makes it possible to do just that. Imagine you’ve made some changes to trunk you want to check in as a single commit (I assume you’ve got the master branch checked out):
git reset --soft refs/remotes/projectA # Reset to the remote HEAD git commit -c ORIG_HEAD # Commit our original HEAD node into this tree.
Voila, the changes will now be combined into a single commit, and the old commits will be gone (well, technically they exist they’re just orphaned). Note, this does change “the past”, and of course fiddling around with history is generally not a good idea. But given we’re talking about a local repository, here, I don’t see a big problem with it.
Of course, a better alternative is to make all your changes in a local branch. Then, when you’re ready, you can merge the changes into the originating branch, and then commit them there. That way, you can opt to delete the working branch if you don’t care about the history, or keep it around if you think you might need it.
So there ya go. Basic git-svn bridging on a non-standard tree layout. And if you’ve read this sentence, congrats! Your ability to stay awake against all odds is, without a doubt, stunning.
Victory is Mine!
Well, I won the battle, anyway, even if the war is still ongoing. Word wrapping appears to work, menu items wrap, larger font sizes are now fully supported. And it looks mighty fine! And so, NetHackDS version 1.10 is now available!
Of course, now I have to move on to the myriad other features on the list.
Slow, Inexorable Progress
Very very slow… or, at least it feels that way. In reality, I can’t really complain. Word wrapping, even nice paragraph-reflowing word wrapping, is in and works quite nicely. Meanwhile, the command window is now finished, and supports snazzy scrolling when it’s necessary (of course, the default font still doesn’t require it, but switching to, say, the map font, which is 8x16, results in four pages of commands). It’s lightening quick, too, unlike the rest of the rendering code (yay large backbuffers and DMA copies). Meanwhile, the menu item wrapping code is… coming along.
Of course, the menu code has been a constant thorn in my side… I’m starting to realize I really need to go back and rethink how the code is designed, but at this point, I just want to get it working and move the heck on. Of course, that may be easier said than done. Currently, the big problem is that, previously, I could assume menu pages were the same size. After all, every item had the same height, so once the page size was calculated, it was basically constant. But now, with items being wrapped, they may take up two or even three lines in the menu, and I suddenly need a much smarter algorithm for determining how to page forward/back through the menu. It’s all quite tedious… which is why, I suspect, I left it for so long. sigh
Update:
Well, in a wave of inspiration, I managed to get the scrolling issues fixed before I left for work this morning (yay for extremely simple solutions). There’s still at least one crasher I’ve come across, but so far things are looking pretty darn good, if I do say so myself.
Word Wrapping For The Win
While development has slowed considerably on NetHackDS, partly due to me play… err… testing, partly due to me just needing a bit of a frickin’ break (and partly due to the excessive drinking I did during our company Christmas party, which left me nonfunctional for much of today), I have been putting some effort into one of the more difficult enhancements I’m doing to the codebase: adding proper word wrapping.
The impetus of all this started with a comment from one of my users who noted that, due to his/her poor vision, they’ve found my port rather difficult to use, as it makes use of a fairly small font. As such, they were wondering if I could add support for larger font sizes, and I, of course, said I’d take a look at it. It also happens that the only major remaining “known issue” in the code is one of text clipping: since the DS can only display, oh, maybe 65 columns of text, while much of the text in the game is designed for an 80-column display, there are a number of instances where the text is just chopped off on the right-hand side. And this is related to the font size issue because both problems require the same solution: word wrapping.
Now, the most basic word wrapping algorithm one might come up with, where you simply chop text where it’s too long, and print the remaining portion on the next line, does the job well enough, but it’s pretty unsightly. You end up getting things like this:
Before:
Hello, this is a really long piece of text I'm using for an example. And here is another line of text.
After:
Hello, this is a really long piece of text I'm using for an example. And here is another line of text.
Ideally, in a situation like this, what you really want is for the word wrapping engine to understand that the text is really a paragraph, and the words shouldn’t just be wrapped, but reflowed as well, so you get something like:
Hello, this is a really long piece of text I'm using for an example. And here is another line of text.
instead. So, being the wildly talented programmer that I am, I reinvented yet another wheel, wrote a reflowing word-wrapping algorithm, and voila! Fancy reflowed text. And it looks mighty fine, too. But there’s a problem. In order to write an algorithm like this, you have to identify paragraphs. Now, it just so happens that blank lines are usually used to separate paragraphs (you need only look at this post to see that), and so it’s pretty easy to identify where one paragraph starts and the other ends. But, NetHack also likes to generate tabular data like this:
No Points Name Hp [max] 1 48117 brettk-Val-Hum-Fem-Neu died in The Gnomish Mines on level 12. Killed by an Elvenking. - [115] 2 39115 brettk-Val-Hum-Fem-Neu died in The Gnomish Mines on level 10 [max 11]. Killed by a vampire lord. - [79] 3 29901 brettk-Bar-Hum-Mal-Neu died in The Gnomish Mines on level 11. Killed by a hallucinogen-distorted giant ant, while helpless. - [105]
and if you identify paragraphs by blank lines, and you reflow text, you end up with:
No Points Name Hp [max] 1 48117 brettk-Val-Hum-Fem-Neu died in The Gnomish Mines on level 12. Killed by an Elvenking. - [115] 2 39115 brettk-Val-Hum-Fem-Neu died in The Gnomish Mines on level 10 [max 11]. Killed by a vampire lord. - [79] 3 29901 brettk-Bar-Hum-Mal-Neu died in The Gnomish Mines on level 11. Killed by a hallucinogen-distorted giant ant, while helpless. - [105]
Notice how the rows are all jammed together. See, to the word wrapping engine, this looks like one big paragraph, thus it wrapped it accordingly, and the result is an unreadable mess. So, in this case, you don’t want to reflow the text. In fact, there’s really no nice way of handling this, so you’re best off just using the ol’ chop-and-print approach, as at least it somewhat preserves the tabular layout. But what the heck do we do, now?
Well, the NetHack putstr() call that is used to write text to a window, and is the entry point I implement, takes an attribute, which normally specifies bold, underline, etc, for the text. So, I created a new attribute bit, 0x1000, which basically says “don’t reflow this text”. I then went into the NetHack core (yeah, more core mods… I don’t like it, either) and altered the topten code so that, for the NDS version, it specifies this “dont reflow” attribute. This same attribute is also used wherever files are paged out (such as the in-game help text). But that does mean that, in effect, I’ve had to special case the NetHack core in a couple places in order to get the correct wrapping effect… basically, the solution sucks. Unfortunately, I don’t know of a better one.
But, on the bright side, the word wrapping works! And I’m using the same code in the status updates, the text windows, the message output, and eventually the menu code. Then, I just need to create a scrollable command window (the current one is entirely static), and the new, non-clipping, larger-font-capable NHDS will be ready! Ish.