Umich AFS

The University of Michigan does not offer support for mounting ITS AFS space from non-university machines. I was able to set it up after a bit, and thought I’d document it here in the hopes that it’s helpful. For Ubuntu, the packages are openafs-client, krb5-user, and openafs-krb5. The realm name is UMICH.EDU – case matters. Next, as the official docs say: run kinit <username>, (assuming your local username differs from your uniqname) which will prompt for your password, followed by aklog. You should now be authenticated as yourself with your home directory at /afs/umich.edu/user/<username first letter>/<username second letter>/<username>/.

Stuff

It seems clear in retrospect that my Windows installation died due to a failing hard drive; I RMA’d it after the self-test reported read failures. I hope to build a RAID array at some point, but it’s somewhat out of my price range currently to buy a couple of drives.

College costs too much – when I talk to people who went to college in the 70s, they tell me how they were able to pay for college with a part-time job. What happened?!

I’m trying to figure out if over the summer (hopefully as part of GSOC!) I could write a filesharing application for Freenet – one that would make Freenet about as easy as Vuze to find and download stuff. User interface would be incredibly important in this effort, as features are not sufficient for  – and too often impede – intuitiveness. My target audience is people who aren’t computer-savvy. I’d want to use a lot of jQuery to make it more like a desktop application. When I made my chat plugin using Toadlets it was somewhat of a pain so I hope I can avoid having to do that somehow. I also need to learn more about and implement proper Model-View-Controller separation. Freenet itself doesn’t really have this. :\ Perhaps I could use WoT and allow people to publish a list of keys to associate with an identity. The tool should make it easy to assemble and update these lists, and people could share keys to the lists (likely USKs?) outside of the application-offered channel (ex Freemail) in order to have secret lists.

Collectd and School

I’ve discovered collectd, which is a pretty comprehensive logging daemon. I’m using the ping plugin to attempt to get some data on when (and hopefully why) the Internet connection here in the apartments starts being worse than usual. The graphs aren’t incredibly clear, and it’s not as simple as lots of packet loss anymore – even so, TF2/Counter-Strike can be unplayable, especially later at night. I’m pinging the gateway on the other side of the DSL connection, as well as a few servers around the US. I wonder if the spikes later on are what’s causing the problem, but I don’t know if that’s due to more network load from us or in general.

My classes are interesting, but I have enough work that I feel like I’m always behind. I like political science though – questions such as what the role of government is in free speech, or what free speech is fascinate me.

Problem Types

I find that among my least favorite types of problems are those that I’m unable to learn from. My main system drive was spontaneously remounted read-only, and upon Alt-Sysreq-reisub’ing, the OS didn’t come up and I got “error: partition not found” and a grub rescue> prompt that couldn’t do anything; not even “help.” I pushed in all the SATA cables and it came up, but upon reboot the drives were out of correct boot order. Bizarre. The part about this that scares me is that I’m for the most part unable to learn anything from this, and I wasn’t able to do anything to stop it from happening again because I don’t know why it happened. The same thing applies to the mysterious times this machine goes completely unresponsive while idle or suddenly doesn’t have video on boot, then spontaneously regains it. The former has happened a few times, the latter only one.

From programming in assembly, I finally realize how segmentation faults are really nice compared to the alternative. Data and instruction separation is a luxury. Miss a bounds check and suddenly you’re executing things not intended to be instructions and you get really weird opcodes and the whole thing dies. It can get really frustrating.

I realized the only reasons my server has gone down at dad’s are due to external forces: either the power has gone out at the power outage or circuit breaker level, or cables have been unplugged by unwitting family members. I wonder how much better colos are. ChunkHost was really nice, and I’d have likely continued once my “free beta” ended (I have my suspicions it’s a marketing thing for “free trial”) if I had more disposable income to the point where I felt I could justify a monthly fee.

EDIT: I had forgotten the time it went down as I was upgrading from Debian Lenny to Squeeze. I had set up a virtual machine for fallback, but I didn’t end up using it: in restoring the VM from backup I unknowingly uncovered a configuration problem with one of the hosted sites that showed up a few days later on the main server. Whoops.

Learning Through Disaster

Gather ’round children, and I’ll tell you a tale of what happened to a Linux box when its sole filesystem was remounted read-only due to disk errors. This coincided with the backup server being taken offline with an errant circuit breaker.

I first became aware of something rotten in the state of Webserver when the sites hosted on it became messes of PHP errors in place of content. I could ssh in, but after entering my password I was greeted with:

-bash: /etc/profile: Input/output error
-bash: /home/steve/.profile: Input/output error
-bash-3.2$

That was a scary greeting.

lighttpd and ssh continued running, but PHP died, commands other than Bash builtins refused to run, and Bash profiles failed to load. I don’t know if some of this is due to damage or the partition being read-only. I’m pretty sure that commands expect /tmp, /var/run, and /var/lock to be writable. I now have those mounted as tmpfs as per the instructions on the Arch wiki here. The warning about lighttpd seems to not apply in my case. cURL seemed to run at first but died when I tried to do anything with it. I had hoped to POST files over, as scp, ftp, and sftp would not run. su still worked. Tunneling worked too, so I was able to still access other machines behind the firewall through the server even though I couldn’t run ssh from the machine itself. I ended up using cat to copy over text files. For binary files I had to get a great deal more creative. The only way I could interact with the server was over ssh; the server was an hour away and even if I did have physical access, mount refused to run (unable to write to /etc/mtab?) and I was afraid that the files I could access might be only buffered in memory and that rebooting into a LiveCD/USB would lose them. My options were limited. I had to use only Bash builtins to pull binary files off the server in text form. I modified a version of this hexdump script to pull files over ssh using | tee file.log to avoid having to copy-paste. tee takes output from stdout and redirects it to stdout and a file given as an argument. Here’s the script:

exec 3<"$1" while read -s -u 3 -d '' -r -n 1 char do printf "%02x" "'$char" done

I lacked a text editor and couldn't write to anything on the root filesystem. I found a tmpfs mount point (I used /lib/init/rw but /dev/shm would also work.) and stored the file by echoing the script line-by-line. In retrospect, I could have used \n and the -e (interpret backslash escapes) option to do it in one line: echo -e "exec 3<\"\$1\"\nwhile read -s -u 3 -d '' -r -n 1 char\ndo\nprintf \"%02x\" \"'\$char\"\ndone" > scriptfile. I ran it with bash scriptfile target_file.

All of this effort, though fun, ended up being unneeded as I had forgotten about my set-and-forget backups. Hooray rdiff-backup!

I ran mysqldump nightly and let rdiff-backup handle any differences. I restored it on the new machine with source mysql_dump.sql on a mysqladmin prompt, but as it contained users and privileges things got messy as the root and debian-sys-maint accounts were partially overwritten. I used mysqladmin to sort out the root password confusion and phpmyadmin to replace the debian-sys-maint password with the one found (in plaintext?!) in /etc/mysql/debian.cnf.

It was a fun puzzle even though it was ill-timed.

The Wolverine Soft 48-hour game competition revealed to me just how difficult physics engines are to make. I spent two days coding and recoding collision resolution only to get different sets of bizarre, game-breaking glitches. At least collision detection was easy because everything was a circle. It was fun and I'd like to do it again. Maybe I should become familiar with a physics library such as Bullet and ask for it to be approved for use in the competition. The guideline is unless it's an approved library, all code and assets (with exceptions for music and sound effects) must be created primarily on-site within the 48-hours. Next time I'll have to plan to do homework in advance. Ignoring homework for a weekend is inadvisable.

I am currently taking 17 credits, and the time management is very difficult, though has not yet proven to be entirely impossible. I'm considering taking classes at LCC this summer to lighten the load during the next school year. I applied to Camp CAEN to be a counselor, but they emailed back saying camp was ending due to the director retiring. The odd part is their website, as of this writing, has no mention of it that I can find. I'll have to see if I can get an internship over the summer.

Update

The power outage made for a very interesting problem. I had inadvertently broken PHP by turning on compression in php.ini when apparently I wasn’t supposed to. This was back when I was trying to get the WordPress compression working. I didn’t know PHP’s configuration wasn’t working until the server was forced to reboot due to the power outage. The initial problem was that the server was not set to turn back on automatically after power loss, which I had set intentionally fearing damage if it attempted to turn on during a brownout, but perhaps I should enable that ability. Even once it turned on, after a fsck (over 330 or so days of uptime I think) it hanged on setting the clock and needed a hard reboot, which necessitated that I go over to the server physically and reboot it. Before this I set the DNS entries to my dad’s house so that I could display an explanation of what had happened, and I changed my password for my DNS service as it had gone too long without being reset, and subsequently forgot to correct it in the call to the update script. I managed to get the DNS fixed, then I found PHP was broken, then in the course of my attempts to fix that, lighttpd refused to start at all. I was unaware that I could only declare one error.log, so it appeared that it was not giving any error output, but it was writing to the last error log I had declared, and I was checking the wrong one. Thanks to incredible help from the folk in #lighttpd on freenode, it’s working again.

In other events, I set up Skype on my Grandma’s machine, and I hope she finds it useful. I’ve been working on Breakout in AP Comp Sci and physics collision is interesting, but I think if I end up making a game I’ll be using a physics library for sure. MIT didn’t accept me, as I anticipated. I’ll be going to University of Michigan, but I still have work to do and forms to fill out on the way.

Power Outage

The site was down due to a power outage. I took the downtime as an opportunity to move the shelving the machines are on a foot or two away from the wall so I can get to the back of them. One of the zombies’ networking didn’t come up properly, even with a /etc/init.d/networking stop and restart, but upon reboot it worked correctly. I wish I knew the underlying cause of that.

WordPress Upgrade, RSI, and the LAN

I upgraded to WordPress 2.7. The admin interface is now really slick. It’s gotten an overhaul and is now a pleasure to use, although it seems like it’s slower.  The main page just took an ENTIRE MINUTE to render! This is unacceptable, and hopefully I can get it sorted out. [EDIT: I enabled WP Super Cache, and it seems better. That’s odd because it didn’t seem to enable last time.] Unfortunately, I forgot, although I understood I was supposed to, disable my plugins before the upgrade. Result: white screen of non-loading death and me panicing. Luckily, Google revealed a very helpful page on how to disable plugins by editing the MySQL database, so now the site is back up.

I was hoping to get into RSI, which is an MIT research program. The first sign was that it wanted people who already knew what they wanted to do for their PHDs. (I seriously almost wrote “PDFs” there.) I checked my PSAT and ACT scores and they didn’t meet the minimum. Even though they said that lower scores could be balanced with strong recommendations and whatnot, I decided not to continue the application process. That saves my teachers writing recommandation letters then. It made me feel strange to request an essay from a teacher.

I’m holding a LAN with 8 people on January 2nd. I say 8 because we would need that many for a full game of Versus Left 4 Dead. Pat might not be able to come due to a New Years thing, but I hope it works out.

EDIT: Hmm… The delay in response time looks like it’s limited to default_socket_timout in php.ini…

pfSense

My network at mom’s is now running off pfSense! I took Zombie 6, gave it a second ethernet card I bought from the school, and installed from the LiveCD. Fairly simple, it even let me figure out which card was which by plugging it into the switch! That was cool. The install was uneventful. It started working! I configured the port forwarding and even some fancy DNS options I’d been looking forward to. The domains I host now go straight to the LAN IP of the server when accessed from within the LAN. I think before that it was going out to AT&T and back. DNS seems much faster, although I haven’t put it through a scientific test, nor do I intend to at this point. I set the pfSense box to query the AT&T DNS servers that the modem was querying, although I’m not entirely sure if it’s doing that. Then it broke. I spent what I’m pretty sure was hours going over the configuration of the modem and pfSense box. Then I turned on the monitor, and it was spamming errors which I now unfortunately cannot remember. Google revealed that it was a problem with the PCI bus and ethernet card I bought from the school. (10/100 Mbits, WAN side, a Gigabit card is LAN side) I took down the machine and moved the card to another slot. It started working again, then failed in the same way. I swapped it with the ethernet card in my sister’s machine. It worked instantly in the pfSense box! After another reboot and some nagging, the other card started working on my sister’s Ubuntu box. I then, after some effort, set my Linksys router to be a switch and wireless access point. I had to set the advanced routing option to router instead of gateway, disable its DHCP server, assign it an IP out of the router’s DHCP range, and plug one of the LAN ports (not the uplink!) into the pfSense router. Hooray! The only problems out of all this are that Xfire file transfer didn’t work when Brad tried to send me a file, although it worked a few minutes later for Pat, so whatever, and that for some reason my SRCDS server can’t be seen from the Internet now. I’ll have to check the pfSense forums when I get time, and if worst comes to worst there’s always commercial support… Zoneclient is awesome. I was able to just point it at the modem connection status page, from which it found and used the IP. Surprisingly easy.

pfSense

I read a large part of the feature list. This part made me sad.

Limitations

  • Only works on primary WAN interface – multi-WAN support is available in 2.0.
  • Can only update one account with a single provider. 2.0 enables the use of unlimited accounts.
  • Only works when pfSense has the public IP assigned to one of its interfaces. If you have a modem that obtains your public IP and gives pfSense a private IP, the private IP will be registered with the provider. In 2.0, there is an option to determine your actual public IP and correctly register it.

Given that I do have a modem that assigns a private IP to whatever is connected to it, I’ll either have to use the unstable version if this stuff has already been added, or just stick with what I’m already using, which is really messy and involves duplicate python scripts in cron. If I get the time and will to do so I might hack the script apart so I only need to run it once and it searches the router more efficiently, but… EDIT: D’oh. I can set the script to check multiple domains. I now only run one instance of the thing.