Often I read things in the morning which spur me to go about three hops deep into the internets in search of a few more bits of information.  The following is an example.

I think I read this on wikipedia somewhere:

There are something like 1.1 million US$millionaire households in canada. [http://www.barclayswealth.com/files/volume5.pdf].

Then I wondered how many people in Canada there were:

The population of Canada is something like 33million [http://www.statcan.gc.ca/edu/clock-horloge/edu06f_0001-eng.htm].

Then I wondered how big the Canadian population was:

The Canadian household has mean size something like 2.5 [http://www40.statcan.gc.ca/l01/cst01/famil53a-eng.htm].

Then I thought about doing the (incorrect) simple calculation to determine the ”average” number of Canadians living in households with an excess of 1.1 US$million in assets.

Then I thought about what it would take to do the correct calculation, obtaining the full tabulated sets of income census data …

… and then I stopped (or maybe my coffee high ran out).

This probably fits into the realm of coffee-fueled-geek-wet-dreams but it does seem like there is a need to move toward a browser specifically for data.  There must be applications that people use in specific fields but is there something attempting to be useful and consistent across fields?  I’m sure someone’s already written about this or is working on it but I’ve just started thinking about it and haven’t found anything.

An ideal application would allow one to browse data sets and pull them into a common form and do quick, fun calculations.  As is stands one would have to navigate various governement and instituational sites, transfer data via copy paste and then do something with it in your setting of choice.  Maybe there is a better way.  I need to find some data trufflers and see what they typically do.  Avoiding, the copy and pasting of various sorts of data is probably a pipe dream … but there must be some sort of free and public interface to each of the governement stats sites? No?  I’m pretty ignorant of all this stuff so I’m going to have a look around and maybe post anything I find.  I can imagine that these ideas are the basis of some theses from the various information sciences.

On another note …

Three things.

• My great uncle died last week.  I only met him a few times but I have very fond memories of the visits.  Too bad we didn’t get to visit more but such is distance and life I guess.  He gave me my first glass of champagne.  It’s never been as good since.
• Lilian, an older friend of my mother’s also passed away last week.  I used to visit her with my mom and go swimming in her lake.  She would serve us scotch and gin and would tell amazing stories about sailing around the world.  I think there was some sort of book detailing her trips in photos and words.
• My mom has sold her house and is moving out this week.  It is the home that I grew up in for 17 years.  Last time I visited the trees had grown so tall they looked strange to me.  I guess it is time.  Goodbye old house!

Three things.

• I need a better RSS reader.  I’m using Google Reader presently and it’s ok but I would like something that allows sub-tagging so that I could navigate through paper->arxives->probability or something like that.  Intersections and unions of tags would be alright but I really just need to hide most of my tags so sub-tagging is pretty much key.  Also, the process of adding rss feeds could be more stream-lined.  My reader list is getting pretty messy.  I’m slightly afraid of it.  Better just stick to the comics section.
• Citeulike is nice but I still haven’t yet found a way to automatically export bibliographies with selected options.  You can grab a bibtex file from http://www.citeulike.org/bibtex/user/<username> but you can’t seem to select options.  Not a huge deal at the moment but soon when I have a script to run my *entire day* it may become important.
• I was thinking of using Twitter for sms-posting names of people that I meet (and whose name I want to remember).  I seem to have a really, really hard time with names and sometimes it’s pretty important for businessy stuff.   When I say it like that, it almost justifies it.  Yeah, twitter.  It’s alright.  It’ll be useful.  Yeah. useful.

There’s a possibility I may be looking to implement some sort of interface (web or otherwise) for various tasks related to statistical models/simulations that I will build. I think, if I did do this, it would be with the next year sometime after handing in my dissertation and before getting a job. I’ve been thinking a bit about what such a project would really need, what I would like, and how to put this together to make it fairly usable to both the mathy maintainer (me) and a potentially fairly wide array of users. Here are my thoughts an notes so far. They probably don’t make too much sense as I haven’t done anything much like this before.

Tasks:

• Data Collection (polling): There is some model that takes a large number of input parameters. Some parameters are “known” by modellers while others are estimated by polling experts. Polls are conducted in a funny way that ranks voters according to some weighting algorithm and computes weighted estimates of parameters based on the weighting scheme. I would like voters to be able to draw curves when answering questions, so some sort of graphical interface would be great … I’ve seen a few java applets that seem to do this reasonably well.
• Data Management and Data Entry (modeller): Given a set of polling data, the modeller computes the set of parameter estimates that they need. They also input the other parameters they need for the model. They hit *save* and name the input-set something and voila, they are ready to run simulations. The modeller can have multiple input sets to test different ranges of parameters.
• Running Experiments/Simulations, Output Management, Post-Processing etc.: Simulations are run and output stored for post-processing and analysis.

Polling through a web-interface seems to make sense. After that, the other two tasks can occur offline. For simple, restricted types of simulations and simulation output I could imagine a web-interface but that would be at very late stage of this hypothetical project.

Ideally, in the input collection stage I would like to offer the ability to produce weighted surveys from any selectable subset of users. This means (I think) finding a nice web-interface to a database. I will likely collect and store these through some sort of xml file. See

• http://www.omegahat.org/StatDataML/
• http://www.w3.org/TR/SRML/
• http://www.ilog.com/products/jrules/documentation/jrules67/rstestsim/rs_rsm_refxmldefine4.html

Levels of users or usage:

• developer (code management)
• admin (full data managment)
• modeller (run models, extract data, manage output data only, create subsets of input data)
• data provider (I’m thinking of data coming from polling experts here)

Only the developer and admin class should have destructive access. It would be nice to somehow allow modellers and providers to maintain accounts and manage their profiles through a reasonable interface. This might only be practical for data providers.

This is probably just a “thoughts to myself” page … but any advice, especially on the polling interface, is welcome.

I only just realized that ctrl-shift-t in firefox opens up your recently closed tabs in consecutive tabs.  This has already proved to be quite freeing for me – I feel like I just learned to skate backwards* or something.

In other news, it was Saint-Jean-Baptiste Day yesterday which means lots of Quebec celebrations.  On Saint-Jean-Baptiste ‘eve’ some folks from my street climbed on top of one of the buildings with about a gazillion fireworks and set them off at a nearly constant rate for about five minutes straight.  The whole street quickly became full of people watching the show and when they finished everyone burst into cheers of appreciation … though I’m not sure that the applause weren’t for the buildings not burning down.  I was actually pretty amazed at how good it was and how long it lasted considering it was just a few folks putting together their personal supplies.  I seem to remember some of the fireworks costing a few dollars a pop.  Yet another magical and slightly dangerous night in Saint-Henri.

*I never actually learned to skate backwards.

Since I’ve reformatted my laptop and have not put my music back on yet, I’ve been mosty listening to podcasts.  BBC has some great short documentaries.  In particular, there’s a 23 minute bit in a series “How crime took on the world”, on the BC bud trade that may be worth listening to.

Part 4 of the series talks about ‘IT crimes’ stemming from groups based in Brazil … the reporter essentially maintains that IT and tech skill is really high in Brazil, making a harbour for ‘cyber’ criminals.  I haven’t exactly found much to the contrary in my brief searches but I’m wondering if the reporter was not so tech-savvy and got things a little bit wrong … I’d be interested if any of the IT folks have any idea about the matter.  It’s pretty interested stuff nonetheless.

I’ve recently upped my usage of Tiddlywiki, now keeping a daily work-journal and writing most of my technical notes in it – something which has significantly increased my sanity levels.  Tiddlywiki is nice because it supports a latex type typesetting using jsMath and the content is stored in the local .html file.   The second point may be an issue when file size grows but I think there is work being done on something more scalable with some sort of proper database.  There is also work being done on serverside Tiddlywikis as well.  There seem to be some projects which support multiuser collaboration but I haven’t yet checked them out.  See here and here.

Hopefully, I can get some stuff done and then actually start buying tickets and planning trips more than a few weeks in advance.  I think it’s looking like, after heading out east, I might have a trip to Vancouver in early or mid September.

I absentmindedly cut back on coffee this weekend and then decided to catch up by having a whole wack of joe sometime between 8 and 9 pm tonight. Not suprisingly, I had trouble getting to sleep and decided to work instead.  I’m thinking now about how many posts begin with some sort of comment on coffee consumption.

Good:

Last Sunday night I went to see a friend of a friend’s band, Kids Eat Crayons, at Petit Cafe Campus. I had heard a few tracks before and thought they might be interesting but I was pretty much blown away with the energy and tightness of the live performance. Sounds like a jazzier, loungier, harder Mr. Bungle – though I wouldn’t say I have listened to more than an album’s worth of Mr. B so maybe the comparison is not great. I think their MySpace selections are more jazzy and low key than most of their stuff. They are pretty much a local band for now it seems, playing a few shows in Quebec and Sherbrooke I think but worth heading to if you get the chance.

Awful:

Maple is a commercial symbolic math software package produced with some sort of affiliation (at least originally) with the University of Waterloo. Maple is awful. The GUI is wretched even after 12 (twelve!) versions. My officemate nominated it for one the worst applications awards on slashdot recently. I don’t think it gets enough use to actually win one of those, but I would definitely second the nomination. The only real alternative is Mathematic which has a much more expensive site license … hence we have Maple at work. Recently, I found what I think is a pretty bad bug in Maple which is costing me weeks of delay trying to work through some calculations another way. The bug is something which produces results as bad as (positive #) times (positives #) = negative. I’ve documented the bug on MaplePrimes but the site is conveniently down now. Nothing like commercial software, where for a price you get the privilege of beta testing. Thanks Maple.

Bad:

While writing this post, Firefox crashed.

Nice:

I had not yet saved the post but to my pleasant surprise, the wordpress form was saved by Firefox (or maybe wordpress) somehow.  At this hour of the night, small things like this make me pretty happy.

 I just came across this at  the lumber room.

http://www.tkb.org/Home.jsp

In particular the flash network thing works pretty well.

Also, at the risk of being topical.

I had a thing on the bbc and had to look for this photo (the pop-up seems to be displaying the wrong page). In principle it’s an unremarkable photo and could be any politician with any rock band. The thing to me that’s so striking is that it really shows the extent to which facial expressions are cultural. Maybe, “the band” was tired that day.