snicknet.net

Work has been zapping all of my time over the last 2 weeks. Amazing how that happens. I’m successfully 2 weeks in. I’ve noticed some major changes since I’ve started. I can definitely see my impact on the network as a whole which is very rewarding. I recently came across a problem that I was having a huge issue solving, and that was automating the upload and download of files from a client’s site. Now, I’ve done this dozens of times before using Perl’s WWW::Mechanize library with little to no problems. This time however I was quickly disappointed with a “You need JavaScript support” response page from the client’s site. I guess I had never tried to automate a JavaScript heavy site before, so I thought little of it and moved onto something else, Curl I thought might have support for this. Short of manually constructing some post backs with much time and little likelyhood of success I dropped that idea. Quickly running out of ideas I turned to full blown programming languages, HTTPUnit for Java. This also lacked Javascript support. Running low on Open Source esteem I turned to Microsoft’s .NET WebContent object. Immediately was shot down again with lack of Javascript support. Just about to concede defeat I came across a small forum where somebody had asked the same question. Turns out that Ruby has a gem (a.k.a. plugin for us non-ruby savy folk) named “watir” that controls  Firefox or IE like a human does. Now this sorta sounds like a macro, which I had looked at prior and had turned down because of the lack of control and feedback. Watir isn’t a macro though, every step through the code can throw exceptions, and you can handle them and report back your findings. It’s actually really fun to watch the code execute because you can see it find the form fields, click them, fill them out just like a human. I put together a small script to simulate logging in and had great success with it. So, for all those out there looking for a way to script a Javascript based web page, watir is a great solution. The only drawback is that since it acts like a human the machine is occupied while performing the script, so I’m going to create an Ubuntu VM to run the script in so I don’t have to waste a desktop with the task. A small price to pay to free up several hours of someone’s day at the office. I’ll let you know how the finished script works with the uploads and downloads.

As for this weekend it was quite fun filled and packed. Friday night we attempted to cutover to our new Dell servers which didn’t end up working out, we had major issues with the MySQL replication that had previously been quite content. As a result we decided to delay the cutover. After that, Rubio’s for dinner, then to the movies to see Transformers 2 which was very good (some quirky stupid stuff, but for a sequel very nicely done). Saturday was an all day geek-a-thon with Izzie. We went to the SoCal Code Camp at UCSD, which turned out to be a bit lackluster. We had envisioned something a little more informative and with more depth. Plus we were the only ones in our age group there. After that we cruised into the Gas Lamp district, had dinner at the House of Blues, and then went to the Sophie 103.7 birthday bash concert at the House of Blues. The concert was excellent. Today, I’m just lounging around. Getting some laundry done, and cleaning up. All in all a nice relaxing weekend after several weeks of non-stop go, go, go.