Friday, July 31, 2009

On the woes of usability (cohorts and indicators)

Finding a solution to the indicator report was a big step for me. However, the best thing that came out of the conversation with Mike this morning was something kind of random. For some reason, during the conversation about hard-wiring the "design indicator report" UI, I realized that we also needed to hard-wire the design of a few other pages, just to get the reporting system out the door and into the hands of users ... who could tell us what they really wanted. Which is the gold at the end of the rainbow. Get your application into the hands of your users and they'll tell you what they want. Hopefully.

So the basic conclusion that Mike and I came to today was that the user interface for the new reporting system should be somewhat hard-wired, at least to start with. We've been fighting with this idea for months now. Every time we've wanted to go down the path of "hard-wiring" a report or one of the reporting components, we've designed a cool new widget maker that would hypothetically handle the general case in a way that most users would be ok with. And so after a few conversations, we start heading back down the path toward a "more generalizable" solution.

I now hate generalizable solutions. They suck. They're ugly, they take too long to implement, and they always break. I'm not smart enough to think of every use case that might ever be needed. I've spent 2+ years working on reporting and I'm tired of thinking about vague/general user stories like "Design a Report". You go Design a Report! I just want someone to ask for a very specific feature, like "Give me the ability to retrieve a count of all patients in the HIV Program at the Rwinkwavu hospital". I can do that. I can implement that and make it look nice. And I can build it in a general-enough way so that only the UI needs to be hard-wired. I just can't (and won't ever again) design another generic UI. Go to hell general UIs.

I'm exaggerating. I'll be back to designing general UIs next week. But, I've found a way to handle some of the reporting UIs in such a way that we can build some very nice features for our users without spending months working on a general UI that looks pretty enough and is 100% extensible.

So to get back to the point. This conversation with Mike got me thinking about a part of the reporting tool that I have been dreading since I first conceptualized it. I'm speaking specifically about the Report Dashboard, the main UI that a user encounters when they want to use the new reporting tools. At first, I wanted to make this a portal page ... a place where users could add and configure "widgets" to allow them to view patient data in a very interactive, easy to use manner. I started thinking about all of the widgets we could have "Graph concept X along datetime, break down by Dimension Y" or "Show 10 most recent indicators" or "Show a truncated table of patients that Provider X is supposed to see this week". Any and all of these widgets would be cool and possibly useful. But no one has every asked for them, so I don't think it would be a good use of time building a generalizable portal framework for widgets that might not ever be used. Instead, we should put some effort into hard-wiring some of these widgets according to a user's specification and then try to find some patterns of reuse in those.


(image stolen from the DHIS web site -- thanks guys!)

I still plan to build this dashboard (I want it to look a little like the image above) but we don't have the foundation within OpenMRS for a portal type user interface yet. We have portlets, but we don't have the portal page that would hold the portlets nor do we have a way to configure the portlet to handle any case. For example, there's no way to turn a cohort or dataset based on any concept into a nice graph. But that time will come. I promise you.

However, what we need to be able to do from within the Report Dashboard is to allow a user, like Cheryl down in Rwanda, to quickly find out how many patients are currently enrolled in our HIV Program during a certain period. The cohort builder is great and gives us a good start to solving this issue. But the cohort builder is still too generic. It requires a user to ask a bunch of questions to answer their overall query. A user is left combining multiple queries to figure out how many Male Adult patients are in the system. Why not hard-wire a UI that allows the user to pick a Cohort and select a few dimensions (gender, age) and click "Run" in order to get a crosstab of patients broken down by these two dimensions.


We can easily handle this kind of query with our the reporting framework (our API is very flexible), but it's not at all easy for a user to put together all of the cohort definitions or logic queries through the cohort builder in order to answer this question. They need to calculate each of these numbers separately and then put together the pieces.

Or take this example.

Find out how many patients were enrolled in the HIV Program in Rwinkwavu between 07/01/2009 and 07/31/2009.

Why not hard-wire the UI to handle this kind of query? Clearly, we don't want to hard-wire the values in this particular example. But we want the general pattern to be hard-wired. We'd still generialize it enough and give the user the ability to specify the four variables: Program, Location, Start Date, and End Date. However, the query is a pattern that we know users will ask a lot.

And when a user comes to us and says: "This is great, but I want to be able to get all patients enrolled in the HIV Program AND On ARVs", we don't have to say "Oh F" and spend weeks trying to make the UI take an arbitrary number of Programs and States. We can simply add a new query to the report dashboard to handle this.

Find out how many patients were enrolled in the HIV Program and On ARVs in Rwinkwavu between 07/01/2009 and 07/31/2009.

While it would be great to handle this generically so that a user can ask any question, any time they want, this isn't at all realistic. However, if we could hard-wire a bunch of questions that users ask all the time across all of our implementations, then we'd be 90% of the way there. In addition, we'd remove the need for all of our users to define the same cohort builder queries over all of these implementations. And we'd get some pretty good feedback about what's missing.

That's not to say we don't want to extend the Cohort Builder to allow users to ask more general questions. We just need to give our users a better foundation to start with.

So what query would you like to run within your OpenMRS implementation?

On the woes of usability (from a developer's point-of-view)

I've been struggling with the UI design and usability for our reporting system for weeks now. We've built most of the UIs for the general administration tools for defining cohorts, indicators, datasets, and reports. However, these UI are the kind of pages that are completely unintuitive to a user: a listing page to display all reports and a details page to allow the user to view/edit the details of a report. We give away too much information, too many options and it's just an overwhelming experience for the user. And it's tiring for a developer. I want to build intuitive interfaces. I do. I want to make things easy on our users. I just don't know how. No matter how much I read about ux (user experience), I just can't seem to get it right.

So this morning, instead of banging my head against the wall for another day, I decided to have a discussion with one of OpenMRS's most gifted developers. No, not Ben Wolfe ... although, there's no doubt that he is a programming genius. No the conversation with with Mike Seaton. I asked him to give me a ring to discuss these issues. He's been working on reporting for our legacy Haiti system for years and knows a thing or two about indicators and indicator reports. I was expecting to have a quick conversation about the following mockup:


But we spend about an hour on the phone discussing the details of indicators and how we need to wire parameters (like start date) from the low level reporting components (an indicator) up through the indicator dataset definition and into the report. The screenshot above isn't terribly complicated, but it requires a little too much knowledge of what the lies underneath an indicator.

But as part of a larger process (creating an indicator report), this UI is just one little piece of confusion pie. At first glance (in my mind), the entire UI for creating the indicator report looks like a messy hierarchical chaining type thingamajig because I have been thinking about how one might design one of these reports from a bottom-up perspective. Here's the process, in a nutshell:
  1. User creates the cohort definition (Gender Cohort Definition with "gender" parameter).
  2. User creates the indicator (# Male Patients with "gender" set to "M").
  3. User creates the dataset definition and adds the indicator to it (My Indicator Dataset)
  4. User creates the report and adds the dataset definition (My Indicator Report).
Four different pages ... four different steps ... just to create a report with a single indicator. Why would you do this to the user? Why would you do this to yourself?

I have no idea.

Don't paint me in the moron category just yet. I knew there was a better UI. I had the image in my head, but I just got flustered with all of the "stuff" that needed to happen in order for that design to work. There's a lot of "stuff" that needs to be configured, configuration options that "should" be available to the user. To be more explicit, for each layer (cohort->indicator->dataset->report) there's some "wiring" that needs to happen in order for a report parameter (say "location") to be passed down through the hierarchy of objects to allow a value like "Rwinkwavu" to be used by the indicator or cohort definition when it's time to evaluate the report.

The thing that Mike and I figured out was that we really don't need to let the user configure everything when designing the report from scratch. At least not at the beginning. Let's give them just enough configurability (add an indicator) to get off the ground.
We can make some assumptions for them (you want the "location" parameter to be . Then, once things are flowing, we can let them dig down and get dirty with the report configuration. So here's the mockup I came up with from those conversations:




Now, I know this isn't revolutionary ... but it's hard to make the kind of assumptions we need to make to hide the complexity from the user. And there's still the issue of designing a somewhat intuitive interface for a user that wants to dig down into the details of each indicator in order to configure, say the "location" parameter to be hard-coded. But we'll get to that feature once there's a need for it. For now, the "edit indicator" feature will have to be left unimplemented. Sorry Mike - I know that wasn't the conclusion we came to, but it feels like the right thing to do, rather than adding one more confusing element to this page. This design feels right.

Let me know what you guys think.

Friday, May 22, 2009

You know it's time to backup your hard-drive ...


... when it starts to make audible "clicking" sounds every 10 seconds.

So, I figure, this is a good time for a backup and for me to finally document the procedures I'm using to clone and backup my hard-drive ... (1) to archive them for posterity (2) to make this easier the next time I need to do a backup and (3) to save some others from having to spend time figuring it out on their own.

So I have three backup options to choose from:

(1) Clonezilla: device-device disk/partition to disk/partition
Disk to disk image using Clonezilla. Since this is a manual process I only do this every month or so. However, this is the thing that will save you when your hard-drive crashes and you need to get back up and running as quickly as possibly. Cloning to a drive with the same specs as your working drive will allow you to swap out the broken drive and swap in the cloned drive without losing a beat. I just realized that I need a third internal drive to write the disk-to-disk image in order to keep one disk that will always work (i.e. what happens if both drives end up in the drink during the backup process).

Here's the process for running a Clonezilla disk-to-disk image (screenshots would be nice, but this is all done via CD so I'm not sure if it's possible without a camera)
  1. Insert Clonezilla CD
  2. Reboot System
  3. Answer the following questions
  4. clonezilla> Which language do you prefer? English
  5. clonezilla> Policy for handling keymaps: Don't touch the keymap
  6. clonezilla> Choose the mode: Start Clonezilla
  7. clonezilla> Choose the mode: device-device disk/partition to disk/partition
  8. clonezilla> Choose the mode: disk_to_local_disk
  9. clonezilla> Choose the local disk as source: sda 200GB Hitachi ...
  10. clonezilla> Choose the local disk as target: sdb 100GB Storage
  11. clonezilla> Set advanced parameters: -g-auto Reinstall grub to target harddisk boot sector
  12. clonezilla> Set advanced parameters: Use the partition table from source disk
  13. And finally ... answer the following paranoid, "just watching your back" questions ... I love this part.
  14. clonezilla> Are you sure you want to continue? y
  15. clonezilla> Are you sure you want to continue? If you go on, the original data exist in the target device will be destroyed!!! Be careful!! Are you sure? y
  16. clonezilla> Do you want to clone the boot loader to sdb? y
  17. clonezilla> Now we will start to clone data to target machine ... Are you sure you want to continue? y
  18. Now go get some coffee or a beer (i.e. Bell's Two Hearted Ale or Dogfish Head 60 Minute IPA)
  19. Relax as your life gets cloned over to another hard-drive.

(2) device-image disk/partition to/from image
This is another option and one that I will likely incorporate soon, just so that I always have an extra image lying around in case something goes terribly wrong with the entire universe and this one external drive is the only thing that is spared. For this option, you basically just create the same image as the disk-to-disk image, but copy it to an external hard-drive (any old backup drive), rather than having to buy an exact replica of your current hard drive. This option just gives you a bit more piece of mind and might save you a few bucks in the short-term (you don't have to buy that second internal drive). But once your current drive goes bad (and it will), you'll need to go out and buy that second drive anyway. This just means that your recovery process will take a little longer (i.e. the amount of time it takes amazon to ship you that new drive).

I want to use all three options (just to have an image lying around in case ... yes ... something goes terribly wrong with the universe). However I still haven't adopted this approach (probably because my 500GB external hard-drive is full of Acronis' crappy backups. I hate Acronis, in case I've never mentioned it before. I'll explain in a different post, but essentially it comes down to "four months", which is the time that elapsed between when I sent in an email request to their tech support and when I received a response. If that had happened once, I would have given them a pass. But it's happened twice and I still haven't heard back from them after my last response (sometime in February 2009). For the record, Acronis is a company I want to love (they are from Burlington, MA) ... but I just can't. And until they get back to me with an apology for their terrible tech support, I will continue to give them mean looks when I pass them in the hallway.

(3) Daily backups with Ubuntu sbackup
Daily backup using sbackup (which I just found out was a Google Summer of Code project from 2005). This is going to be the next weapon I add to the backup arsenal, right after I finish my most recent disk-to-disk image.

Conclusion
So I need to get a new external hard-drive. Does anyone have any recommendations regarding backups? I'm thinking about going with a deal from Amazon that includes:
  • Western Digital My Book 1TB USB
  • Western Digital My Passport 320 GB USB
However, I'd love a recommendation. CNET and PCWorld have some recommendations, but nothing looked all that impressive.

And does anyone out there make backups of their backups? I'm thinking about getting an external drive with a few TB of space in order to provide some redundancy in case one of the backup drives fail. So would love some suggestions on that front too.

Friday, May 8, 2009

Update: OpenMRS Test-a-Thon (May 16 - 17)

What the heck is the OpenMRS Test-a-Thon?
The OpenMRS Test-a-Thon is a code-a-thon event that allows people to collaboratively increase and improve our testing process in order to make the OpenMRS platform a more stable and robust piece of software (click here for more details).

When is it going down?
We'll be starting at noon (EDT) on Saturday, May 16 and ending at noon (EDT) on Sunday, May 17. You can come and go as you wish. I'll be hanging out in our IRC Channel all day Saturday and will be taking in a feature film on Saturday night (9 - 11pm CDT). So no pressure. Come inside, submit yourself a junit test, and then go back out and finish your Saturday chores.

Where is the event going to take place?
The test-a-thon will take place online. You can join us on the OpenMRS IRC Channel. Contact me via Skype (jmiranda) or on the IRC channel to discuss how you might be able to help out.


Who can participate?
Anyone. Developers, implementers, supporters, documenters ... people who are bored.

Who should participate?
Everyone.

Why should you participate?
Because ...
  • You love OpenMRS.
  • You want to learn about test driven develement (TDD).
  • You want to learn about behavior driven development (BDD).
  • You want to learn about cool unit testing frameworks (i.e. JUnit).
  • You want to learn about cool web testing frameworks (Selenium).
  • You want to learn about continuous integration.
  • You want to write documentation for OpenMRS.
  • You want to meet the team.
  • You want to make Ben Wolfe happy.
  • You want to win a prize.
  • You have nothing better to do on a Saturday.
How can I participate in this once in a lifetime (more like bi-annual) event?
You can head over to the OpenMRS wiki for more details.

Thank you,

The OpenMRS Team

Warning: DO NOT INSTALL Subclipse 1.6 for Eclipse 3.4.x

The Problem
Save yourself a few days of headache ... don't do it! More details to come.

So Mike (mseaton) and I recently upgraded to Eclipse 3.4.2 because of some messed up issues with JSP/XML Validation within Eclipse 3.4. We also both installed Subclipse 1.6 thinking it would bring us joy. Three days later, with lots of back-and-forth collaboration on a project that we have sitting in Subversion, we've experienced nothing but headaches. When he's committed code to the repository, I'll update my working copy only to find that the only changes that come through are ones that are non-conflicting. So when I then try to commit my changes, I get warnings that my working copy is not up-to-date. I think the problem might be the fact that the new "SVN Update to HEAD" feature does not do a --force (but that's just a hunch). I just finished reverting my Eclipse upgrade to Subclipse 1.6 and have successfully reinstalled Subclipse 1.4.7.

Now the issue is that Subclipse 1.6 (which uses Subversion 1.6 wc format) has upgraded all of my working copy metadata to Subversion 1.6. Installing Subclipse 1.4.7 (which uses Subversion 1.4 wc format) means that I cannot use Subclipse on my currently checked out code. Since I have about 30+ projects checked out, this is kind of a problem. I can't waste hours

Here's the error you'll get when

root@jmiranda-laptop:~/Workspace/module-birtreport# svn diff
svn: This client is too old to work with working copy '.'. You need
to get a newer Subversion client, or to downgrade this working copy.
See http://subversion.tigris.org/faq.html#working-copy-format-change
for details.


So thankfully, the subversion folks have a downgrade script:

http://subversion.tigris.org/faq.html#working-copy-format-change


Download the script and run the following command:

root@jmiranda-laptop:~/Workspace# python change-svn-wc-format.py module-birtreport --force --verbose 1.5
Converted WC at 'module-birtreport' into format 9 for Subversion 1.5


Now, at least, I can run svn commands on that directory.

root@jmiranda-laptop:~/Workspace/module-birtreport# svn diff
Index: .project
===================================================================
--- .project (revision 7566)
+++ .project (working copy)
@@ -1,18 +1,18 @@
...

However, when I open Eclipse I can no longer see the very helpful icon overlays on top of files that are "unchanged" or "dirty". I just get unadorned, naked file icons.

Anyone have any ideas on how to fix it?

"Taste my sad, Michael" -G.O.B Bluth

Update:
So the "icon overlay" was looking more and more like an Eclipse issue, so I decided to start Eclipse with the "-clean" option.

/usr/lib/eclipse3.4.2/eclipse -clean


That fixed it. And still a few hours short of what it would have taken to reinstall Eclipse, plugins, and re-download all source code.

"Taste the happy, Michael" -G.O.B Bluth

Update:
It's still broken. Only the top-level folders were successfully restored after the -clean restart. After a few restarts and rebuilds, none of the files or folders have the icon overlay. So we're back to square one.

Following Ben's advice, I tried to install Subversive, but this did not help either. This seems to be a problem with Eclipse. It's as if Eclipse is blocking the icon overlay feature of any plugin (except the core plugins that mark files with "build errors". Checking the properties of any of the files definitely shows that they are "managed" and should have at least the "managed" icon.

One last thing (check the error logs again) and then I'm going to re-install Eclipse.

Last Update:
Ok, so some combination of the above instructions, as well as closing all projects and dealing with them one at a time, while drinking coffee, and threatening to reinstall Eclipse while shaking your fist at your computer, on top of the Celtics winning Game 2 of the Eastern Conference Semifinals has seemed to fix the problem.

I can't explain it folks. Just keep playing around until it works. The one project at a time helped me focus on what I was doing.

(1) Run the script to convert from svn 1.6 to svn 1.5
(2) Clean project
(3) Build project
(4) If that fails to work, close Eclipse and restart with the (-clean) option.

The last thing I was going to try was to create a brand new workspace and move one project over at a time (running the svn convert script). It didn't come to that thankfully.

As I mentioned earlier, the last resort option would have been to reinstall Eclipse and download all of my project source code. Given that half of my day was spent troubleshooting this, I would recommend going that route when you're confronted with this issue and can't get it resolved within an hour.

Friday, May 1, 2009

Walk MS 2009: Part 2

This is my mom (the one whose head is being covered by my giant hand). Her name is Claudia Angel Miranda. She's 56 years old. And she has Multiple Sclerosis.

Her middle/maiden name is no accident -- she had to be an angel to put up with the crap that we put her through when we were kids. But aside from possibly being an angel, she is also one of the greatest moms I've ever known. Heck, she's one of the greatest people I've ever known. Her strength, kindness, sensitivity made me the person I am today. I love the heck out of this woman.

My mom started having symptoms of Multiple Sclerosis in her mid-20's. She was diagnosed with MS around her 30th birthday. Multiple Sclerosis is one of the more confusing and misunderstood diseases out there. You need an M.D. just to understand the Wikipedia article about MS. But here are some facts:

There is no cure.

There are no known causes.

It physically and mentally debilitates people.

It affects each person in different ways.

It is painful to cope with MS.

It is painful to watch someone you love cope with MS.




This is a difficult subject for me - I’ve never been comfortable talking about my mother's condition. And it's never been easy for me to ask for help. However, I've realized over the years that my silence is not helping and that it’s extremely important for me to begin talking about MS in order to make more people aware of the disease.

Last year, with the help of friends, family and co-workers, I raised $3000 (ranking in the top 100 for the Wisconsin Chapter of the MS Society).

Thank you to everyone who donated last year.

This year, Megan and I are hoping to raise $5000. Yes, I realize that's quite a bit of dough, but $3000 seemed like a lot last year and we did it!

We don't need large donations. We just need lots of small donations. So donate $1, $5, $10, $25. It all adds up.

Donate $1, $5, $10, $25

OR

Join us for the MS Walk in Madison

Thank you so much for reading.


Friday, April 17, 2009

Walk MS 2009: Part 1

In a few weeks (Sunday, May 3rd 2009), Megan and I are going to be participating in the 2009 MS Walk in Madison, WI. I’m blogging to ask you all for a donation (of any size) to help in the fight against Multiple Sclerosis. Last year, with your help, we raised: $2500.00. Not bad for our first year.

But this year we want to double that. For the mathematically challenged, that means we're looking to raise a whopping:

$5000.00

So please help us achieve this goal by
making a tax-deductible donation to the National Multiple Sclerosis Society! Any amount will help. Give $1, $5, $10, $20. Every dollar counts.


Thank you for your support,

Justin and Megan


Some useful links: