Download the World

Clojure Report Generation

2010-11-15T21:07:00.008-05:00

The entire code from this blog post is here. I will chunk it up to explain:
https://gist.github.com/4fcd64dc020a9c749b76

Working with collections in Clojure makes programming such a natural experience. Especially when you want to report on your data. Many different data stores have libraries which make getting information into Clojure trivial. They often use a collection of maps where map keys are the "column" names and the values are the data. The following generate-report function should work on any collection of maps.

Above I am taking a set of query results (i.e. a collection of maps) and any number of report items. A small sample of one of the maps in the collections with which I'm dealing is here.

The first thing to do is grab the headers. These are just the firsts of each report item. Pretty simple. The process function takes the keys from each report-item and applies manipulation-function to the result of (get-in % ks). Process returns a new function that takes a query-result, so I can map it over every report-item to get columns as a seq of functions. Finally to get rows, I use apply juxt to call every function in columns on each query-result to get a vector of each of the results of those calls.

You can tell from the doc string what report-items are supposed to look like, so I will show you two of my reports. The first one uses the generate-report function to put its :headers and :rows into an html table.

The code above is all that is required to create this:

Granted the manipulation functions in each report-item are doing most of the display work, the generate-report function makes it easier to focus on what matters out of each map in the collection of query-results.

The reason I wanted to break out the generate-report function is that reports on the same data might not always look the same. The next example looks at a report that needs to export to a spreadsheet. For the HTML table, I wanted to shorten up some of the fields, by displaying "Lastname, First initial." The title tag for the td displays the entire name if the user needs to know. However, once exported, there are no title tags, so the user would have no idea if he was looking at John Smith's or Jessica Smith's score. In xls-report I use a function to display all of the parts of the operators and evaluators which together constitute a unique identifier (see employee-display).

This report demonstrates the use of the nested maps by getting out the vector of [:client :account-numer] and [:client :account-name]. These are passed to a get-in call which gets me the :client of each score, and then the :account-number and :account-name of that result. Now the *.xls that gets saved can be sorted by either account-number or account-name.

Definitely handy stuff. I'm really starting to love this whole idea that code is data and vice versa.

clojure map-keys

2010-11-04T11:28:00.004-04:00

Someone in #clojure was asking about how to case-insensitively check the keys of a map. The best answer anyone present at the time came up with was pre-processing the map by naming, lowering, and keywordifying the keywords. We then found out the asker was looking for keys in nested maps, so I wrote a function for that.

After writing the function I realized I could extract the lowering function and let it take a function to apply to all keys of m and all keys in m's vals.

Here is the map-keys function with the lower-key function extracted. I run it on some goofy data at the end. Try it in a REPL.

Clojure Dates with Java Interop

2010-10-08T22:26:00.003-04:00

I really just want to test how an embedded gist looks in blogger. I thought I would do it with a simple function I wrote to test if a date object was between two other date objects in clojure. I found how to do it in Java on stackoverflow, so I took advantage of clojure's java interop, and this is what we get:

The Idea

2010-10-03T23:40:00.003-04:00

I'm standing in my low basement. I'm on one of few marked spots in which I can stand up straight--between the floor rafters. Standing up straight aims my sight directly at a pair of old computers. They aren't incredibly old, but given the nature of the technology world and the fact that they were going to be thrown away, we can call them old.

However, with the addition of some spare parts from other "old" computers, I have on my hands two 2.6 GHz computers with 2 GB RAM and 40 GB HDD in each. Not so bad for "old" computers. Then it hits me: why don't I try out mongodb's replica sets. Then I start brainstorming all of the possibilities, and I think, why don't I document all of this on the web?

I have been learning Clojure this year. It's a language in which my boss wrote a great wrapper around the Java mongodb driver called karras. Since April I have been putting together a web application in Clojure which uses karras and mongodb. At MongoBoston this year, I watched a lot of great talks on interesting features of mongodb. Also, I'd like to learn more about deploying redundant applications.

Anyway, here's what I want to do. Run two physical machines with Ubuntu server. Within each of these host two virtual machines. With the four total virtual machines, create a mongodb replica set and a fallback to keep my application live if either physical machine goes down. The point of documenting my plans here, is to get (hopefully constructive) feedback which will help me build a fairly robust system on two very cheap machines.

I will set some time for installing ubuntu server on both computers the next time I go in the office. Between now and the time this actually happens I have some questions to anyone who feels knowledgeable on the subject.

Can I even do what is described above?
Would it be better to oust the VMs simply using 2 instances of mongod on each physical machine?
What would you do differently given the hardware mentioned above?

Optimize your Optimization.

2010-09-26T22:21:00.002-04:00

I constantly am looking for a better way to write my functions. I want them to be fast, flexible and readable. However, in the fast department, I also want to write them fast. Tradeoffs are always made in programming. It hit me recently to optimize my optimizations. That is, focus my optimization on the things used the most.

I was working out a function in a web app controller which gets accessed every time there is a write operation on the database. I was trying to figure out how to perform less reads to get the data into the write function. This is when I realized that the write operations would be done, on average, less than once a day. I think I can handle two or three reads per write in that frequency.

Now I can move on, but while I'm working on optimizing things that happen every second, what do you think of this approach? Am I caring too little? Should I be concerned of anything? How would you approach it?

33,000 cases of beer on the wall. . .

2010-09-26T00:27:00.001-04:00

SHSH - home - BONHEUR PROVISOIRE

so beautiful

2010-09-21T22:24:00.002-04:00

It's so beautiful to see success stories like this:

No Arms, No Legs, No Problem for English Channel Swimmer

2010-08-24T21:05:00.000-04:00

It turns out that the Calvin and Hobbes complete collection makes the perfect "table" for a two player game of skip bo. Win.

2010-08-24T20:59:00.000-04:00

looks like i need a new camera

2010-08-24T20:57:00.000-04:00

Pretty amazing.

Best In Class: Clojure vs Ruby & Scala - Transient Newsgroups

2010-07-07T21:03:00.000-04:00

Best In Class: Clojure vs Ruby & Scala - Transient Newsgroups

This was great. I keep seeing so many reasons that Clojure is awesome. It's no wonder the language is growing so fast. I want to take this approach, but instead of searching the file contents, I want to display all images in a directory in chronological order inside of a web interface. So I can run something like this:

clj slideshow.clj "/path/to/my/photos"

then visit the directory's url http://localhost:8080/photos (photos being the dirname from above) to see an ordered slideshow of my images with some neat jquery effects. The best part about the jquery is that the images are housed locally, so they should load extra fast, making the effects run smoothly. In fact, rather than a command line argument it could just launch the server with a page that asks for the path of the photos directory.

anyway, that's what i'm going to try before jumping right into an official work project as my first clojure application.

Flickr: jabella's stuff tagged with balloonlaunch

2010-03-19T23:46:00.001-04:00

I totally want to do this. Flickr: jabella's stuff tagged with balloonlaunch

I've noticed that I share more links than anyone. Either that, or none of my friends allow me to see links they've shared in the privacy settings. But it makes me wonder, why would my friends read so many uninteresting things? If it's interesting, share it. Please?

My Desk At Work

2010-03-11T15:19:00.002-05:00

Rearranged my desk after getting a new monitor. The one on the right is hooked to a Windows desktop where I do all my stuff that requires windows. The one on the left is hooked up to my MacBook where I do a lot of coding and emails. Everything you see here (besides what's on the shelves) belongs to notifymd. I love my job.

Built a desk at home out of an old door we removed during rennovations to the house. I'll post pictures of that set up when it's done.

adsense

2009-03-29T21:12:00.002-04:00

i just set up this blog to have adsense, because it was really easy and why not? i saw the following ad while i was checking it out to make sure it all worked and looked nice.

Do you like blogging? This could be important.

Spoiler follows. You should really watch it first.

Turns out, it's a sweet Dentyne commercial. Not really that sweet i guess. I am just impressed with the fact that they surprised me to see a gum commercial targetted at blogger nerds (no offense blogger nerds; i am one of you).

git on windows 7

2009-01-28T16:15:00.002-05:00

going forward i think i will use this as a link holder for when i find useful links and don't want to forget them. why hog the usefulness for myself?

here is the first one:

git on windows 7

code

2008-12-31T12:44:00.002-05:00

i'm looking for a code highlighter for my other blog that has python syntax highlighting. i plan to post a lot more code snippets on there.

i thought google might have something so i was checking out my old blogger. guess not.

fossils

2007-12-19T21:36:00.000-05:00

fossils suggest whales evolved from tiny deer-like creatures

does anyone think that some of the creatures that are only 'known' because of fossils could be in the imagination of some paleontologist who didn't actually find all the 'pieces' for which he was looking? give it some thought. seriously, some of those short t-rex arms could have been missing two segments or something. okay, so that example is a bit ridiculous, but i just think people find something, and their findings get exaggerated until some new creature 'evolves' out of the discovery of an old giraffe neck-bone laying next to a turtle shell.

i have a theory

2007-12-08T12:49:00.000-05:00

There is a new product on Gmail that will change the world. AIM on Gmail is going to create more people being down with Gmail. Once all the people are using it, they can phase out the AIM part, because everyone should have realized that Google rules, and Gmail is the shit. I am going to be the biggest advocate of Google besides the people at Google themselves. If anyone needs a Gmail account to see why I'm nerding all over the place, send me a message with your email address. If you don't want to put it on here, you had to have seen this link on AIM, Gmail, or Facebook so message me on that.

In other news, I heard a radio commercial for a place that said, "Come on in and find out why we're the best kept secret in town." This is very puzzling, because I can't understand why they would be a secret if they were that good. It was a furniture store or something, so I can't see why people would keep the place a secret. Is there anyone trying to keep this furniture store for himself? I don't think so. It's a secret, because the place sucks, and nobody cares.

Goodbye.

are you serious?

2007-11-15T02:37:00.000-05:00

unbelievable

A 17-year-old was accused of and arrested for stealing virtual furniture out of a resort in some networking site. I understand he was hacking, and that is the illegal part. Here is the part I can't believe: people PAY to have virtual furniture. Over $5000 dollars worth of virtual furniture was "stolen."

Can anyone at all tell me the point of this?

Go Boston Red Zox. . . World Champions!

2007-10-29T01:33:00.001-04:00

about scrabble--

eben: i hope i get the z and the x.

brandon: what would you spell?

eben: i don't know. . . zox.

PS. the sox have won the world series twice in my lifetime. also, they stopped the yankees 11 year pennant winning streak. here's to many more rings for boston.

outrageous halloween stuff

2007-10-25T02:08:00.000-04:00

these pictures are absolutely ridiculous. enjoy. . .

Starburst

2007-10-20T12:27:00.000-04:00

This commercial is so awesome. Thanks Starburst marketing.

halloween.

2007-10-18T23:01:00.000-04:00

halloween is on it's way, and i have nothing to wear. i never think of good costume ideas until after it's over. by the time it comes around the next year, i forgot all my ideas. i'm bummed. let me know what i can be.

you bet, if it's aspercreme

2007-09-11T21:28:00.000-04:00

that is what you would hear if you heard a commercial for aspercreme today. however, before that lame slogan, they had this catchy one:

you bet your sweet aspercreme. this ad only ran for a limited time. i caught it a handful of times before they changed the slogan. apparently you can say shit on tv, but you can't say sweet aspercreme. pish posh.

in other news, subway has had a scrabble game going on this summer. i have a theory that, as sad as it is, facebook's scrabulous application plays a huge part in this. besides a small handful of people, no one played scrabble leisurely. now both subway and facebook are promoting scrabble hardcore. maybe milton bradley is in cahoots with subway and facebook and that's the real reason this is going on. whatever, i guess. i still love scrabble.

update: while searching for how long the commercial actually ran (i estimate, based on the comments, around three months), i came across a ridiculous post that is completely the opposite of what i was saying. go tell this guy what a douchebag he is. and all his friends.

i felt it!

2007-09-07T11:41:00.000-04:00

ian and i laughed at this commercial for years when we saw it years ago. in other words, we are still laughing at this commercial now. last night, in portland on fessenden st where the parties apparently happen out of nowhere on a thursday night, when not even some of the people who live there know what just happened, an event occurred that sent us cracking up, and no one would understand what was going on. anyway, someone at the beer pong table was getting really pumped and excited about a rebuttal. in getting pumped he said, "something magical has to happen here." shortly after that, something good for his side did indeed happen, and he proclaimed, "something magical is happening here! Did you feel it?!" When no one chimed in, he answered himself with, "i felt it" in the exact tone that janice says it above.

so yeah, the point is that it would be really impressive to see a movie in which the characters develop inside jokes throughout the movie, and those jokes come up again later in really funny situations. like the time that friends and i heard the exact same song in a driver's ed instruction video that we saw in a porno a couple years earlier.

going for a walk

2007-09-04T22:41:00.000-04:00

i am about to go for a walk around farmington. maybe scope out the help wanted signs in the windows, since i am probably going to be in farmington a lot this fall. ian and i helped eben move most of the stuff that is in his new apartment. it's a pretty sweet place to chill. it's really big too.

i haven't really taken advantage of this new blog, i apologize to anyone who pays attention. i think jesse is the only one. my plan was to use my book to write down creative ideas to put in here. instead, it turned into a lyric book in combination with a movie idea book. lame i know, but i think my friends are funny people, and i would really like to create a movie with some of the great dialog that i hear every day. you know, i've never gone an entire calendar day without laughing. in my whole lifetime i can't think of one single day i did not belt out a hearty laugh.

anyway, since my book is getting filled, i will let everyone in on some of my ideas. for now, i will leave some lyrics i thought of and liked a lot. maybe later i'll ask for your opinions on my movie ideas. for now, just read and give me advice on this stuff.

oh yeah i have no title for this yet. . .

pawned most of her stuff
guess i didn't love her enough
besides, that camera had too many memories of us

looking back on signs
really should have realized
but, i left her thirteen hundred miles behind

now as time passes
it forces memory lapses
although, i remember looking better in these sunglasses

it needs a chorus, but i hope you liked it. i know i did. i imagine the band rocking out after each verse with it quiet while the lines are given.

bye for now.

new one again

2007-06-21T15:38:00.000-04:00

apparently my old blog was a trial. so it's not free, and i'm not paying. so here i have a blogger. i will link to eben and willie soon, as i am almost sure they are both on blogger. alright so i am heading the hell outta here to do stuff. be in nh for the rest of the month without a computer. call me. i don't even know if i will have a place to plug in my phone. so if i don't answer, leave me a message. i will get back to you in july.