Mockups and Life with a VM

Well, while my machine is in the shop. I’ve been doing some mockups of the new UI changes in OLM. I don’t consider myself a photoshop guru but Martin showed me the wonders of Ctrl + Shift + J, which takes a selection and cuts it out into a new layer; making mock ups a lot easier to do. Being a perfectionist when it comes to photoshopping slowed me down a little, but I think the final product paid off and looks quite nice (of course, who wouldn’t think their own work of art isn’t amazing), below are some of the screenshots thumbs:

Of course, the fun part is over, once we decide if the look is good or not, we shall implement.. which will take a bit more time and a lot more thinking. But I am a little excited about how OLM is going to look after we are done with the summer work.

Now, my machine is in the shop and well… I do like to get some productivity done when I’m done photoshopping… (i.e. for the most part, now), so while photoshopping yesterday, I decided to finally start playing with VMware Fusion.. which is still in beta, and I must say, is pretty awesome. I’ve installed parallels before but didn’t like it too much, mostly because by default, it doesn’t set your host to become a router, but rather, your VM has its own mac address… the result? I can’t use UTORwin on campus! garrr…

Now, before the Parallels Guru comes knocking down my door and asking “do this this and that and voila its done! you’re dumb for not thinking of it”. I consider myself to be an average joe for the most part, if it isnt intuitive and the defaults settings aren’t set to the way I like them, and it takes more than 5 seconds to change something. I’ll probably find another program to see if they are better.

The good news being, although reinstalling Ubuntu the first time was utter hell and took me a few days to setup OLM. Installing Ubuntu on VMware and installing OLM with all the packages and stuff took me … about 4 – 5 hours this time, a big improvement :)… I think I should time myself, maybe aim for the 1 hour mark 😛

Tony

The Art of Re-installing

So, the new hard drive is in; and right off the bat, its not crashing as much.. well, it crashed once so far, but I’m not sure if thats out of sheer luck or something. But an interesting 4 hours of headache with some Unicode encoding and PyXML.

It was bizzare, I started off the day going “ok, some stuff seems to be broken, just simply reinstall it”; since I had my share of playing around with Ubuntu and linux by now, I figure this was going to be a easy task, except when I tried installing TurboGears.. apparently, python complained and won’t let me.. Sure, I checked my Macbook and my macbook has Python 2.4.4 and my desktop has 2.4.3, so I might as well upgrade it… and thats when all hell went broke loose.

I’ve gotten used to building projects from source now (something which I wasn’t when I first started), So I downloaded Python 2.4.4 source and build it from scratched; worked fine. Installed pyXML, and TurboGears and all lights are green. Then I tried running OLM…

ImportError: From some path:  undefined symbol: PyUnicodeUCS4_Decode

Ok… some googling later and it tells me to build the source using the option -enable-unicode=UCS4

So I did… and it had a hell of a time installing.. or lack of, it didn’t install at all.. always breaking halfway. 2 hours of headache later, I googled up 2.4.3, which wasn’t obvious on the Python website, but it is there. Installed it and and wow.. everything worked… I’m relieved, but dammit.. why the hell didn’t it work earlier!

Going in circles with source compiling, never versions aren’t always the better version.

Tony

Paths paths paths…

So my work machine has a tendancy of crashing.. so I’m relying less on it right now (since my productivity hits a wall when I’m trying to debugg and everything dies). So I went on a quest… (at least, until my machine is fixed) to install OLM on my Apple Laptop.. needless to say, it was a lot more work and headscratching than i anticipated.

Karen managed to get it running relatively quickly, and I figured out after a day, that I needed to set a few more paths and copy some files over for TurboGears to see SQLAlchemy. Of course, I don’t believe a day of playing around should really discourage anyone from installing OLM on OS X, so I wrote an entry on the wiki, hopefully speeding up the installation 😛

HTML encoding function thingy…

Well, I would first like to say, thank you to everyone who answered my last blog entry. CONCAT was a good idea but we are trying to keep it valid within the SQL standard, anything specific to postgresql or such we would like to avoid.

Well, I manage to stumble onto another problem. This time, we are at the javascripts calling my python function (if that makes any sense).

My function in python takes an element, which is either a String, or a List itself. It first check to see if it is a list, doing this with type(args) == type([]). If its not, then it assumes its a string, and processes it.

To my knowledge, i can call my function “release” by doing:

self.user.goto(“/release?myList=something”)

which will return a string, If i want a list, I would do:

self.user.goto(“/release?myList=somethign&myList=somethingelse”)

The problem arises when I try to input an empty list… I thought about trying:

self.user.goto(“/release?myList=&myList=”)

but that returns a list of 2 empty strings inside 😦

UPDATE: So I figured it out.. made me almost smack myself for not thinking of it.. apparently, passing in nothing will give me an empty list, exactly what I’m looking for 🙂

SQL Woes.. Multi-fetch!

The problem:

We are refactoring and cleaning up on OLM , an online marking tool. As part of that, we’ve switched from SQLObject to SQLAlchemy, but we are having some performance problems related to the number of queries. Our navigation page displays students’ names, grades, and so on. It also shows the TA who is marking their assignment. The problem arises when there is more than one TA marking one student’s assignment.

For example, suppose we have a student named Bob, who is being marked by Jane and Liang (both of whom are also students). We have one query which gets all the TA’s marking Bob. In the navigate page, under the markers for Bob, we wish to put a list of TAs, like this:

Name	Grade	Marker
Bob	B	Jane, Liang

Unfortunately, we can’t think of a way to get all of the TAs
marking a particular student as a single string with a single query.
Instead, we are looping, which hurts performance a lot.

The Schema:

User
user_id	int, primary key
user_name	char
student_number	char
display_name	char
created	date
groups	foreign key to Groups

—

Status
status_id	int, primary key
state	string
grade	float
assignment	foreign key to Assignment
author	foreign key to User
overall_comment	char
lock_holder	foreign key to User
lock_timestamp	date

—

PermissionMap
permission_id	int, primary key
author	foreign key to User
marker	foreign key to User
assignment	foreign key to Assignment
permission	foreign key to Permission

The Query:

To acquire the student and his grade we do:

select distinct d.user_name, d.status.grade
from User natural join Status as d
where d.user_name = d.author

To acquire the markers list:

select PermissionMap.marker
from PermissionMap
where PermissionMap.author = our_student

This would give us all the markers, and then we would parse it and manually put it into the table.

Well, as you can see, our goal is to remove the code required to shove the marker list into the marker column, is it possible to do the whole thing within SQL? Also, if we have 300 students, would be nice if we didn’t have to do 300 different queries to the database.. is there an elegant solution to this that we are missing?

Tony

Programming.. in the real world *gasp*

Me as a Programmer
My experience as a programmer was mainly as a random hobby or doing it as part of an assignment for school. While I have a strong interest in programming, I admit I haven’t indulge myself into any big projects where I have to do a lot of poking around. Most of the courses I’ve taken as an undergraduate student so far involves little or no starter code, and you build the thing from scratch.

Introduction to the world of a programmer, OLM
It didn’t take me long to realize how lost I was when I started with OLM. My first task was a simple one, update the wiki and include information on any tools needed for the developers. A guide was already there for checking out and setting up the project, though it was missing a few tools like the python profiler and the development kits of a few packages.

It was when I got OLM up and running and started adding stuff to the code, was where my lack of experience searching through things kicked in. I didn’t know where everything was, the tools were all foreign to me and I certainly have no idea how everything gets called. All this while Martin was blazing away installing the profiler and finding out why our navigate page took 3 seconds to load.

The first line of code I added was about 4 – 5 hours into the analyzing. By the time I finally finished my 2nd warm up (which was augmenting to the navigate page to show the rubric breakdown), it took about 8 hours and maybe 40 lines of code.

It took me about a week before I was comfortable using javascript and SQLObject, which then made augmenting the code a lot speedier. I was closing tickets a lot faster (the easy ones anyway), though I might have gotten carried away with some things and committed too early without really testing out all possibilities. At first, I submitted really buggy code… further playing around revealed random bugs that I didn’t expect to happen when I written them. Although I thought this is where I can finally start blazing through the code, changing things left and right; Martin had come up with some interesting discoveries

Speed Issues
It turns out that the template engine we were using (kid) was rendering pretty slowly, having only 100 students, rendering the page with their names, marks and such took 3 seconds.. adding the rubric augmentation only exploded this result, making it into 10 seconds. With this, we fired off an email to the creators of TurboGears hoping they might know why our pages were rendering so slowly.

Switching to SQLAlchemy
It was pretty bad, we got an email back from the creator of TurboGears and they tell us that they don’t even use SQLObject anymore, they switched over to SQLAlchemy. Through a series of more meetings, we decided to switch over to SQLAlchemy.

The syntax of SQLAlchemy was a lot different than SQLObject, it also didn’t help that the documentation online wasn’t as helpful with very limited examples. A lot of head scratching and a week later, I’m still a little shaky working with SQLAlchemy, though it is getting better. In the end, this is far better than than typing out raw SQL queries.

I have learnt a lot so far and done a lot of relearning. I haven’t touched python in a year but after using it, never programmed with Javascripts before and didn’t know what a template engine was. I am doing a bit better now, though I still have lots to learn, especially with using debugging tools.

What is OLM?
http://www.drproject.org/olm

Tony