Recipe for Debugging

How do you take a working script and convince yourself it is broken?

1. Try to use outdated sub-scripts in another directory
2. Don't check the test inputs as a potential error
3. All of the above.

In other words, the CSV generating scripts finally have headers! Better analysis here we come. The other reason for the victory dance atmosphere on the project is due to the fact (knock on wood) all the scripts are working and we're collecting the last run of GPU-aware workloads!! The end of data collection is finally nigh. I'm sure I'l regret this phase ending when I have to sit down and write about the analysis. Meantime it wouldn't be unreasonable to feel a little burn out on stalking data logs.

Moving forward at last!

It's not always finding the big bugs that lead to breakthroughs on research. Sometimes it's the little ones that trail into different problems and at long last the real cause of all those headaches is found. Last semester the scripts gave a strange error during one of the test runs to collect data. Killall had nothing to kill. Strange, according to the machine collecting data the machine under test was suppose to be in the middle of running the disk at a particular frequency and collecting the AC measurements once every minute. Curious, I was sent off to comb the logs.

The logs revealed a bit of oversight and assumption coming back to bite us squarely you know where. The two computers time stamps were not in sync. Not just the hour was off but the minute, second, they had drifted apart by several minutes. Ubuntu checks the time of a computer against an external server only on boot up. We leave rickroll and lolcat running all the time. Not a problem, server administrators the world over solve this problem of time drifting before. Google to the rescue and help from the lovely folks on the Ubuntu community forums later, we had a quick fix. Thankfully this fix was quick and easy, add a line to the chron.daily directory. Chron is a unix utility that runs the scrips inside its directories at a set interval like daily, hourly etc.

Wait overnight. Come to find the timestamps don't match the next day. Move the fix to the chron.hourly directory. Repeat waiting. No dice the next day. Discover the network time protocol daemon hadn't be installed! Install ntp, wait over night. I did a little dance the next morning to see rickroll and lolcat's timestamps. They actually stayed in sync!

Now once Vince finishes off his script to parse our data logs into a format easier to read, we'll be able to take data from a run and actually analyse it. Of course the first model we create with testZulu will be ugly, but at least then we'll be moving forward!

Cannot touch `cupcake': No such file or directory

Yes, calibration script, no cupcakes for you. Instead, victory cupcakes for me because I made it work! Nevermore did I want to dance, cheer and celebrate seeing that line. It meant my wonderful script from last week (touch the log, and test the directory path headache) was finally working. I can actually start looking at a new problem within some other file. Any file. Maybe calib again, so long as I don’t have to go near that block of code for a while. Why? Coding and patience are fragile things.

If that wonderful ssh connected terminal, happens to have a poor internet connection, well, you’re out of luck to debug the script as you test without serious lag. Frustrating but there’s workaround hacks like write the code offline, connect, run the test, look at the bugs, disconnect, repeat. Even better, right as the solution dangles itself to you to solve all problems once and for all, said internet connection DIES.

It’s like playing a game when Windows wants to restart for an update. Windows keeps minimizing your game right as you’re battling the big boss bad guy to bug you “Oh hey do you want to restart now or later for these updates to take effect?” No amount of patience will save you from the rage quit. Write down the genius solution. Take a day off. Pray to the campus dorm internet idols for a better connection & ping rate, then go at that sucker like there is no tomorrow. Code in anger. It gets the job done. Especially with enough lies, like if you finally get it working you never have to touch that code again.

Error between the Keyboard & Brain

My main task this week was to expand the calibration script with a simple task I’d done
before with a slight twist. First create a file using touch to hold a log in the directory for
a certain test, set up an if case because the user might want the log in a separate directory
and ask the user which they want to use. We’d rerun a few tests and then finally start
looking at some data. I had a few other tasks on top of this, but obviously I did not get
to those because programming can be deceptive. Once you feel grounded in another
language it can lead to a blind trust in the code you write. Hubris edging you on the
whole time, Pssh we’ve been using unix for two years now. We’ve used touch a thousand
times on the command line. So easy.

Instantly then when things go wrong we must distrust the new language! Dude no way
did I mess up my unix commands. It’s all perl’s fault! I’ve spend hours on the script, read
online tutorials finding no one else had problems doing this, reread the man page on
touch. It’s all stupid perl!

Except when perl isn’t at fault. And I realized this hours later. When my cutesy
workaround hack began turning into a contender for an obfuscated perl contest entry.
After shooting off an email to my professor that made me feel crummy. I obviously
sucked at coding for not solving this problem on my own. It’s so simple. Then the
response came and she asked if maybe the directory path wasn’t correct in my test inputs.
That reply almost made me ready to hide under my desk because that hinted at hours of
backing up mentally and rewriting the whole block of code to my original solution.

I would have loved to let myself get mad at that jerk of a user. All that time wasted! Bad
input was at fault! My code isn’t suppose to catch the user’s errors! But I was that bad
test user and I let those errors make me distrust the 80% working code. A little tweak to
get it 100% working was all it would take.

But really this stumble reminded me I’ve only been coding seriously for barely two
years. I’m still very much a toddler in terms of coding experience. This setback won’t
discourage me. Until this giant mess, I didn’t notice the huge improvements from my
first Programming I class to now. The important thing I need to keep reminding myself
is I need to keep writing consistently correct code over the long haul. That’s all that
is required. Not rushing myself because I should be handling more and writing harder
scripts. The code’s going to show that rushing by not working.

BIOS, cpufreq, perl, sprinkles & cherries on top

The Undergraduate Research Commandments

by Stephanie Schmidt

1. Thou shalt do a lot of work.

2. Thou shalt have nothing to show for it more often than not.

3. Everyone in research shall pass through this phase.

4. Lie to thyself. Call it a phase even though thou shall know it will happen again.

5. And again.

6. And again.

When in the course of research, your allotment of tasks become disjointed from one another, remaining organized is crucial. Otherwise you won’t be able to avoid the problem of tasks falling through the cracks. This applies to groups of any size. As often as people stress this point, it’ll continually get blown off by fellow undergraduates as just one of those things seasoned professionals say to scare you. Er yeah...the folks brow beating this point, same ones who cry over bad documentation, once again have a valid point.

I promise for most schemes it’s not just busy work. Having the time to get through all the tasks per week is a different story entirely. Understanding the blog platform you are using is another good idea too, especially if posts go missing. Excuse this attempt to recapture the essence last week’s post covered. Debugging scripts, tweaking BIOS settings and learning the fun of redundant work sounds like an awesome week with so much progress made, eh? Quite.

Debugging should have its own class somehow in undergraduate requirements. I don’t care that people have their own method. Every debugging method boils down to the same handful of steps; what differs is the artistic way the steps are executed. Debugging a big file you didn’t write is scary. Especially when you’re a total n00b at the language. Just those ten lines I used to expanded a function brought with it a whole heap of warnings and bugs thrown up on the formerly working-ish program. Good documentation and cout statements were pretty much all that saved my sanity. Google is a great tool until you don’t know how to properly phase the bug in general terms.

The frustration iceberg was trying to find a compromise between perl variable syntax and command line expressions. Working around that one turned into hours of creative reworking to avoid any need for awk. Changing $1 to /$1 may be great for telling perl to not touch that variable. The command line, however, looking at that fix will act five year old being asked what is the square root of negative three. Failing silently is the worst problem a debugging entails. Wait, spending hours elbow deep in Google trying to figure out why an entire command line isn’t being grabbed only to learn oh...it’s not actually the code you wrote. Someone else’s piece is acting weird. Good documentation folks: not just a lame requirement but a means to cutting that debug phase in half if you can rule out bugs coming from pre-existing code.

A result of all this real world style programming has been a major change to my debugging method. I’ve started documenting outside of my code what steps I take to debug. It helps me revert on the fly back to old settings without bulk commenting out changes. Just because sometimes it won’t be your lines code that is causing the barf of bugs. Oh and updates? Don’t necessarily trust them, they can mess things up too. I’m looking at you Apple: ios4 has been a mistake to older generation devices.

Step back from the programming side of things. Appreciate how simplistically elegant the BIOS interface is for navigating. Getting into the BIOS itself can be an hour of frustrated restarts, mistimed keys & not hitting the correct key. Hint: it’s always delete. Except when its escape. Or when it’s F11. At least the restart button is universal. I give mad props to the BIOS interface though, because by enabling SpeedStep my data collecting life became easier. cpufreq actually started sending legit verbose data! cpufreq happens to be directly linked to a component within our benchmark software, which in turn is used when examining the collected data. Nothing important to hurry and get working you can imagine.

My tasks this week often had nothing, or very little, to show for themselves given how long they took to execute. Patience was key for jumping to the next task. Patience to integrate new information debugging brought and sucking up the pain of opening yet another instance of google or terminal. Patience is the hidden side to research. And lies. Lying to yourself that all that hard work gone unseen this weekend did something will be easier to accept under the lies of sprinkles & cherries on top.