In order to improve the performance of mysql when dealing with large databases, there's a couple of settings that can be tweaked in your my.cnf file (or through a MySQL administration tool). The following line has proven to be the most useful.
innodb_buffer_pool_size = 1G
I've been meaning to write about this improvement for the last few weeks, but I wanted to test it myself first. So far I'm very pleased with the performance improvement (database restore time is down from about 3 hours to about 45 minutes on a ~1.2G database). I haven't tested how this setting affects MySQL during normal execution, but it certainly helps improve performance during backups and restores. So if you find yourself backing up and restoring large MySQL databases, you should test this out. You should (obviously) not set innodb_buffer_pool_size to a value greater than the available memory on your machine.
Thanks to Marc Harrison for the tip.
If you want to read some more about this configuration, I found a couple of blog entries from a few years ago Enjoy!
http://www.mysqlperformanceblog.com/2007/11/01/innodb-performance-optimization-basics/
http://www.mysqlperformanceblog.com/2007/11/03/choosing-innodb_buffer_pool_size/
Thursday, December 17, 2009
Wednesday, December 16, 2009
On setting up a development environment in Mac OS X 10.6 Snow Leopard
I just bought the new Mac Mini from Apple. It's my first Mac since the Mac Plus of the early 90's and I have to say ... it feels good to be back. I still love doing development on Ubuntu, but Mac OS X is easy to get used to and a lot of fun to play around with. Nothing compares to "apt-get install ..." (on Ubuntu) when getting a development environment set up, but this tutorial for Tomcat 6 probably saved me a couple of hours of agony of going it alone.
Thursday, December 3, 2009
Fixing an existing subversion project to migrate to the trunk/branches/tags convention
Here's the procedure I used to move an existing "dumb" subversion project to one that follows the subversion convention (trunk/branches/svn). It's a combination of the two ideas presented in the articles below. More steps then I was hoping for, but it's a pretty simple-to-follow solution and much less aggravating than the Eclipse approach.
(this procedure assumes that you are using Linux, but the process would be similar on Windows or Mac)
// 1. Checkout a fresh copy of your project
$ svn co http://svn.myproject.com/myproject myproject
// 2. Change directories to make it easier perform svn operations
$ cd myproject
// 3. Add the new trunk directory
$ svn mkdir trunk
// 4. Locate all files/dirs in the current directory and svn move them to the trunk directory
$ find . -maxdepth 1 ! -name trunk -exec svn mv '{}' trunk \;
// 5. Double check that all files have been svn move'd (.classpath, .project, build.xml, etc)
$ ls -la trunk
// 6. Commit all changes
$ svn ci -m "myproject: moving all files from myproject into myproject/trunk"
// 7. Switch your local copy of the module to
$ svn switch http://svn.myproject.com/myproject/trunk myproject
Resources:
(this procedure assumes that you are using Linux, but the process would be similar on Windows or Mac)
// 1. Checkout a fresh copy of your project
$ svn co http://svn.myproject.com/myproject myproject
// 2. Change directories to make it easier perform svn operations
$ cd myproject
// 3. Add the new trunk directory
$ svn mkdir trunk
// 4. Locate all files/dirs in the current directory and svn move them to the trunk directory
$ find . -maxdepth 1 ! -name trunk -exec svn mv '{}' trunk \;
// 5. Double check that all files have been svn move'd (.classpath, .project, build.xml, etc)
$ ls -la trunk
// 6. Commit all changes
$ svn ci -m "myproject: moving all files from myproject into myproject/trunk"
// 7. Switch your local copy of the module to
$ svn switch http://svn.myproject.com/myproject/trunk myproject
Resources:
Thursday, October 29, 2009
OpenMRS Developers Conference (Oct 2009)
OpenMRS Developers Conference in Latin America
The following document was used by the developers/trainers at the OpenMRS Developer's Conference in Lima, Peru. We kept track of the agenda and topics we discussed during the week in order to possible create a curriculum for future training sessions. I'd be glad to discuss this with anyone who is looking to create training manuals for both short and long-term training sessions on OpenMRS-related technologies.
This document is still pretty raw, but feel free to use bits and pieces if you find them useful.
http://docs.google.com/View?id=dnc9s5q_10gb5qb4dn
The following document was used by the developers/trainers at the OpenMRS Developer's Conference in Lima, Peru. We kept track of the agenda and topics we discussed during the week in order to possible create a curriculum for future training sessions. I'd be glad to discuss this with anyone who is looking to create training manuals for both short and long-term training sessions on OpenMRS-related technologies.
This document is still pretty raw, but feel free to use bits and pieces if you find them useful.
http://docs.google.com/View?id=dnc9s5q_10gb5qb4dn
Monday, September 21, 2009
Extending the OpenMRS Reporting API: Cohort Queries
Introduction
At the Cape Town meeting last week, we demo'd (if you could call it that) the new reporting framework and some of the early versions of tools that we've built on top of it. Some of those tools have been described in recent entries and others are still undocumented. That will change soon. The UI and tool set for the reporting framework is still immature, but we're happy with what we've come up with from an API standpoint. As a developer, you *should* be able to do most of what you need with what exists. However, there will be instances where you need to extend the framework to do what you want.
NOTE: If anyone knows a good plugin to include code snippets in Blogger, please let me know.
Extending the Cohort Query
Here's a quick example of how to use the API to evaluate a cohort definition. This will return you a list of patients that can be used as input into another piece of code.
This is a pretty trivial example, but there are lots of cohort definitions in the system that will allow you to do more complex queries. With that said, there will likely be a time when you need to implement a cohort query that is not currently supported within the framework.
For instance, let's say that the Ministry of Health within the country you work calls your mobile and says "Hey, Justin ... we need to report on the number of patients that have had the XYZ form completed on a certain date or in the past month/year".
In order to extend the system to support this query, we need to do three things: (1) write a new cohort definition to define this query, (2) write the evaluator to evaluate the cohort definition and return a cohort.
0) Write a JUnit test that tests what your new cohort definition should do
1) Write the cohort definition
So the first step is to define the cohort definition. We just need to extend the DateRangeCohortDefinition to allow dates to be specified as properties or parameters and add a "forms" property that will be used to allow a user to select forms in the system.
2) Write the cohort definition evaluator
I want to use the API (instead of writing my own DAO, so we have to do something slightly stupid here. The PatientSetService.getPatientsHavingEncounters(...) method only takes a single form as a parameter, therefore I need to iterate over the forms in the cohort definition and call the API each time. If performance was an issue (and if I had more time) I would write a DAO that does the actually query I want. But neither of those statements are true, so this is a perfectly good solution.
3.) Now, let's can go back and run our JUnit test ... and hopefully see green.
At the Cape Town meeting last week, we demo'd (if you could call it that) the new reporting framework and some of the early versions of tools that we've built on top of it. Some of those tools have been described in recent entries and others are still undocumented. That will change soon. The UI and tool set for the reporting framework is still immature, but we're happy with what we've come up with from an API standpoint. As a developer, you *should* be able to do most of what you need with what exists. However, there will be instances where you need to extend the framework to do what you want.
NOTE: If anyone knows a good plugin to include code snippets in Blogger, please let me know.
Extending the Cohort Query
Here's a quick example of how to use the API to evaluate a cohort definition. This will return you a list of patients that can be used as input into another piece of code.
// Define the cohort definition
GenderCohortDefinition gcd = new GenderCohortDefinition();
gcd.setGender("M");
// Evaluate the cohort definition
Cohort males = Context.getService(CohortDefinitionService.class).evaluate(gcd, new EvaluationContext());
This is a pretty trivial example, but there are lots of cohort definitions in the system that will allow you to do more complex queries. With that said, there will likely be a time when you need to implement a cohort query that is not currently supported within the framework.
For instance, let's say that the Ministry of Health within the country you work calls your mobile and says "Hey, Justin ... we need to report on the number of patients that have had the XYZ form completed on a certain date or in the past month/year".
In order to extend the system to support this query, we need to do three things: (1) write a new cohort definition to define this query, (2) write the evaluator to evaluate the cohort definition and return a cohort.
0) Write a JUnit test that tests what your new cohort definition should do
public class FormCohortDefinitionEvaluatorTest extends BaseModuleContextSensitiveTest {
private Log log = LogFactory.getLog(this.getClass());
@Test
public void evaluate_shouldReturnPatientsWhoHadFormCompletedOnDate() throws Exception {
FormCohortDefinition fcd = new FormCohortDefinition();
fcd.setForms(Context.getFormService().getAllForms());
Calendar newYears = Calendar.getInstance();
newYears.set(2009, 0, 1);
EvaluationContext evalContext = new EvaluationContext();
evalContext.addParameterValue("sinceDate", newYears.getTime());
evalContext.addParameterValue("untilDate", newYears.getTime());
Cohort cohort = Context.getService(CohortDefinitionService.class).evaluate(fcd, evalContext);
Assert.assertEquals(1, cohort.getSize());
}
}
1) Write the cohort definition
So the first step is to define the cohort definition. We just need to extend the DateRangeCohortDefinition to allow dates to be specified as properties or parameters and add a "forms" property that will be used to allow a user to select forms in the system.
public class FormCohortDefinition extends DateRangeCohortDefinition {
private static final long serialVersionUID = 1L;
@ConfigurationProperty(required=false)
private List<Form> forms;
public FormCohortDefinition() {
super();
}
public List<Form> getForms() {
return forms;
}
public void setForms(List<Form> forms) {
this.forms = forms;
}
}
2) Write the cohort definition evaluator
@Handler(supports={FormCohortDefinition.class})
public class FormCohortDefinitionEvaluator implements CohortDefinitionEvaluator {
/**
* Default Constructor
*/
public FormCohortDefinitionEvaluator() {}
/**
* @see CohortDefinitionEvaluator#evaluateCohort(CohortDefinition, EvaluationContext)
*/
public Cohort evaluate(CohortDefinition cohortDefinition, EvaluationContext context) {
FormCohortDefinition fcd = (FormCohortDefinition) cohortDefinition;
Date fromDate = fcd.getCalculatedFromDate(context);
Date toDate = fcd.getCalculatedToDate(context);
Cohort cohort = new Cohort();
PatientSetService pss = Context.getPatientSetService();
for (Form form : fcd.getForms()) {
Listtypes = null;
Cohort tempCohort = pss.getPatientsHavingEncounters(types, null, form, fromDate, toDate, null, null);
cohort = Cohort.union(cohort, tempCohort);
}
return cohort;
}
}
I want to use the API (instead of writing my own DAO, so we have to do something slightly stupid here. The PatientSetService.getPatientsHavingEncounters(...) method only takes a single form as a parameter, therefore I need to iterate over the forms in the cohort definition and call the API each time. If performance was an issue (and if I had more time) I would write a DAO that does the actually query I want. But neither of those statements are true, so this is a perfectly good solution.
3.) Now, let's can go back and run our JUnit test ... and hopefully see green.
INFO - LoggingAdvice.invoke(102) |2009-09-21 17:13:46,399| In method CohortDefinitionService.evaluate. Arguments: CohortDefinition=null, EvaluationContext=org.openmrs.module.evaluation.EvaluationContext@1d742a1,
INFO - LoggingAdvice.invoke(127) |2009-09-21 17:13:46,528| Exiting method evaluate
INFO - FormCohortDefinitionEvaluatorTest.evaluate_shouldEvaluate(51) |2009-09-21 17:13:46,536| # of patients: 1
Friday, August 7, 2009
On the Cohort Dashboard (implementation)
So here's the first pass at an implementation of the Cohort Dashboard. The only thing that is currently working is the fact that we can pass this page a cohort. The rest of the functionality still needs to be implemented. Let me know what you think.
Thursday, August 6, 2009
On the Cohort Dashboard (design)
This is the page that will complement the Cohort Builder. The page will be accessible by users who want to analyze a set of patients from the top-down. More details to come ...
Tuesday, August 4, 2009
On the Reporting Dashboard (implementation)
Updated (10 August 2009): Added a new screenshot.
Here's the latest verison of the Reporting Dashboard with some working charts and a more intuitive Cohort Breakdown widget.
Here's my initial version of the reporting dashboard, using JQuery's portlet library. Let me know what you think. Some of the fields (program, workflow, cohort, dataset) are populated dynamically, but there's nothing on the backend (yet) to process most of these. Writing that code will be relatively simple once I hear back that this is the right direction to take the dashboard.
Here's the latest verison of the Reporting Dashboard with some working charts and a more intuitive Cohort Breakdown widget.
Here's my initial version of the reporting dashboard, using JQuery's portlet library. Let me know what you think. Some of the fields (program, workflow, cohort, dataset) are populated dynamically, but there's nothing on the backend (yet) to process most of these. Writing that code will be relatively simple once I hear back that this is the right direction to take the dashboard.
Friday, July 31, 2009
On the Reporting Dashboard (design)
Here's a quick mockup of what I'm thinking the Report Dashboard should look like. I need some help with some of the most common queries that users ask about. You'll quickly notice that the dashboard is missing a graphical widget. Once I think of something to implement, I will add it. Let me know if you have any suggestions.
On the woes of usability (cohorts and indicators)
Finding a solution to the indicator report was a big step for me. However, the best thing that came out of the conversation with Mike this morning was something kind of random. For some reason, during the conversation about hard-wiring the "design indicator report" UI, I realized that we also needed to hard-wire the design of a few other pages, just to get the reporting system out the door and into the hands of users ... who could tell us what they really wanted. Which is the gold at the end of the rainbow. Get your application into the hands of your users and they'll tell you what they want. Hopefully.
So the basic conclusion that Mike and I came to today was that the user interface for the new reporting system should be somewhat hard-wired, at least to start with. We've been fighting with this idea for months now. Every time we've wanted to go down the path of "hard-wiring" a report or one of the reporting components, we've designed a cool new widget maker that would hypothetically handle the general case in a way that most users would be ok with. And so after a few conversations, we start heading back down the path toward a "more generalizable" solution.
I now hate generalizable solutions. They suck. They're ugly, they take too long to implement, and they always break. I'm not smart enough to think of every use case that might ever be needed. I've spent 2+ years working on reporting and I'm tired of thinking about vague/general user stories like "Design a Report". You go Design a Report! I just want someone to ask for a very specific feature, like "Give me the ability to retrieve a count of all patients in the HIV Program at the Rwinkwavu hospital". I can do that. I can implement that and make it look nice. And I can build it in a general-enough way so that only the UI needs to be hard-wired. I just can't (and won't ever again) design another generic UI. Go to hell general UIs.
I'm exaggerating. I'll be back to designing general UIs next week. But, I've found a way to handle some of the reporting UIs in such a way that we can build some very nice features for our users without spending months working on a general UI that looks pretty enough and is 100% extensible.
So to get back to the point. This conversation with Mike got me thinking about a part of the reporting tool that I have been dreading since I first conceptualized it. I'm speaking specifically about the Report Dashboard, the main UI that a user encounters when they want to use the new reporting tools. At first, I wanted to make this a portal page ... a place where users could add and configure "widgets" to allow them to view patient data in a very interactive, easy to use manner. I started thinking about all of the widgets we could have "Graph concept X along datetime, break down by Dimension Y" or "Show 10 most recent indicators" or "Show a truncated table of patients that Provider X is supposed to see this week". Any and all of these widgets would be cool and possibly useful. But no one has every asked for them, so I don't think it would be a good use of time building a generalizable portal framework for widgets that might not ever be used. Instead, we should put some effort into hard-wiring some of these widgets according to a user's specification and then try to find some patterns of reuse in those.
I still plan to build this dashboard (I want it to look a little like the image above) but we don't have the foundation within OpenMRS for a portal type user interface yet. We have portlets, but we don't have the portal page that would hold the portlets nor do we have a way to configure the portlet to handle any case. For example, there's no way to turn a cohort or dataset based on any concept into a nice graph. But that time will come. I promise you.
However, what we need to be able to do from within the Report Dashboard is to allow a user, like Cheryl down in Rwanda, to quickly find out how many patients are currently enrolled in our HIV Program during a certain period. The cohort builder is great and gives us a good start to solving this issue. But the cohort builder is still too generic. It requires a user to ask a bunch of questions to answer their overall query. A user is left combining multiple queries to figure out how many Male Adult patients are in the system. Why not hard-wire a UI that allows the user to pick a Cohort and select a few dimensions (gender, age) and click "Run" in order to get a crosstab of patients broken down by these two dimensions.
We can easily handle this kind of query with our the reporting framework (our API is very flexible), but it's not at all easy for a user to put together all of the cohort definitions or logic queries through the cohort builder in order to answer this question. They need to calculate each of these numbers separately and then put together the pieces.
Or take this example.
Find out how many patients were enrolled in the HIV Program in Rwinkwavu between 07/01/2009 and 07/31/2009.
Why not hard-wire the UI to handle this kind of query? Clearly, we don't want to hard-wire the values in this particular example. But we want the general pattern to be hard-wired. We'd still generialize it enough and give the user the ability to specify the four variables: Program, Location, Start Date, and End Date. However, the query is a pattern that we know users will ask a lot.
And when a user comes to us and says: "This is great, but I want to be able to get all patients enrolled in the HIV Program AND On ARVs", we don't have to say "Oh F" and spend weeks trying to make the UI take an arbitrary number of Programs and States. We can simply add a new query to the report dashboard to handle this.
Find out how many patients were enrolled in the HIV Program and On ARVs in Rwinkwavu between 07/01/2009 and 07/31/2009.
While it would be great to handle this generically so that a user can ask any question, any time they want, this isn't at all realistic. However, if we could hard-wire a bunch of questions that users ask all the time across all of our implementations, then we'd be 90% of the way there. In addition, we'd remove the need for all of our users to define the same cohort builder queries over all of these implementations. And we'd get some pretty good feedback about what's missing.
That's not to say we don't want to extend the Cohort Builder to allow users to ask more general questions. We just need to give our users a better foundation to start with.
So what query would you like to run within your OpenMRS implementation?
So the basic conclusion that Mike and I came to today was that the user interface for the new reporting system should be somewhat hard-wired, at least to start with. We've been fighting with this idea for months now. Every time we've wanted to go down the path of "hard-wiring" a report or one of the reporting components, we've designed a cool new widget maker that would hypothetically handle the general case in a way that most users would be ok with. And so after a few conversations, we start heading back down the path toward a "more generalizable" solution.
I now hate generalizable solutions. They suck. They're ugly, they take too long to implement, and they always break. I'm not smart enough to think of every use case that might ever be needed. I've spent 2+ years working on reporting and I'm tired of thinking about vague/general user stories like "Design a Report". You go Design a Report! I just want someone to ask for a very specific feature, like "Give me the ability to retrieve a count of all patients in the HIV Program at the Rwinkwavu hospital". I can do that. I can implement that and make it look nice. And I can build it in a general-enough way so that only the UI needs to be hard-wired. I just can't (and won't ever again) design another generic UI. Go to hell general UIs.
I'm exaggerating. I'll be back to designing general UIs next week. But, I've found a way to handle some of the reporting UIs in such a way that we can build some very nice features for our users without spending months working on a general UI that looks pretty enough and is 100% extensible.
So to get back to the point. This conversation with Mike got me thinking about a part of the reporting tool that I have been dreading since I first conceptualized it. I'm speaking specifically about the Report Dashboard, the main UI that a user encounters when they want to use the new reporting tools. At first, I wanted to make this a portal page ... a place where users could add and configure "widgets" to allow them to view patient data in a very interactive, easy to use manner. I started thinking about all of the widgets we could have "Graph concept X along datetime, break down by Dimension Y" or "Show 10 most recent indicators" or "Show a truncated table of patients that Provider X is supposed to see this week". Any and all of these widgets would be cool and possibly useful. But no one has every asked for them, so I don't think it would be a good use of time building a generalizable portal framework for widgets that might not ever be used. Instead, we should put some effort into hard-wiring some of these widgets according to a user's specification and then try to find some patterns of reuse in those.
(image stolen from the DHIS web site -- thanks guys!)
I still plan to build this dashboard (I want it to look a little like the image above) but we don't have the foundation within OpenMRS for a portal type user interface yet. We have portlets, but we don't have the portal page that would hold the portlets nor do we have a way to configure the portlet to handle any case. For example, there's no way to turn a cohort or dataset based on any concept into a nice graph. But that time will come. I promise you.
However, what we need to be able to do from within the Report Dashboard is to allow a user, like Cheryl down in Rwanda, to quickly find out how many patients are currently enrolled in our HIV Program during a certain period. The cohort builder is great and gives us a good start to solving this issue. But the cohort builder is still too generic. It requires a user to ask a bunch of questions to answer their overall query. A user is left combining multiple queries to figure out how many Male Adult patients are in the system. Why not hard-wire a UI that allows the user to pick a Cohort and select a few dimensions (gender, age) and click "Run" in order to get a crosstab of patients broken down by these two dimensions.
We can easily handle this kind of query with our the reporting framework (our API is very flexible), but it's not at all easy for a user to put together all of the cohort definitions or logic queries through the cohort builder in order to answer this question. They need to calculate each of these numbers separately and then put together the pieces.
Or take this example.
Find out how many patients were enrolled in the HIV Program in Rwinkwavu between 07/01/2009 and 07/31/2009.
Why not hard-wire the UI to handle this kind of query? Clearly, we don't want to hard-wire the values in this particular example. But we want the general pattern to be hard-wired. We'd still generialize it enough and give the user the ability to specify the four variables: Program, Location, Start Date, and End Date. However, the query is a pattern that we know users will ask a lot.
And when a user comes to us and says: "This is great, but I want to be able to get all patients enrolled in the HIV Program AND On ARVs", we don't have to say "Oh F" and spend weeks trying to make the UI take an arbitrary number of Programs and States. We can simply add a new query to the report dashboard to handle this.
Find out how many patients were enrolled in the HIV Program and On ARVs in Rwinkwavu between 07/01/2009 and 07/31/2009.
While it would be great to handle this generically so that a user can ask any question, any time they want, this isn't at all realistic. However, if we could hard-wire a bunch of questions that users ask all the time across all of our implementations, then we'd be 90% of the way there. In addition, we'd remove the need for all of our users to define the same cohort builder queries over all of these implementations. And we'd get some pretty good feedback about what's missing.
That's not to say we don't want to extend the Cohort Builder to allow users to ask more general questions. We just need to give our users a better foundation to start with.
So what query would you like to run within your OpenMRS implementation?
On the woes of usability (from a developer's point-of-view)
I've been struggling with the UI design and usability for our reporting system for weeks now. We've built most of the UIs for the general administration tools for defining cohorts, indicators, datasets, and reports. However, these UI are the kind of pages that are completely unintuitive to a user: a listing page to display all reports and a details page to allow the user to view/edit the details of a report. We give away too much information, too many options and it's just an overwhelming experience for the user. And it's tiring for a developer. I want to build intuitive interfaces. I do. I want to make things easy on our users. I just don't know how. No matter how much I read about ux (user experience), I just can't seem to get it right.
So this morning, instead of banging my head against the wall for another day, I decided to have a discussion with one of OpenMRS's most gifted developers. No, not Ben Wolfe ... although, there's no doubt that he is a programming genius. No the conversation with with Mike Seaton. I asked him to give me a ring to discuss these issues. He's been working on reporting for our legacy Haiti system for years and knows a thing or two about indicators and indicator reports. I was expecting to have a quick conversation about the following mockup:
But we spend about an hour on the phone discussing the details of indicators and how we need to wire parameters (like start date) from the low level reporting components (an indicator) up through the indicator dataset definition and into the report. The screenshot above isn't terribly complicated, but it requires a little too much knowledge of what the lies underneath an indicator.
But as part of a larger process (creating an indicator report), this UI is just one little piece of confusion pie. At first glance (in my mind), the entire UI for creating the indicator report looks like a messy hierarchical chaining type thingamajig because I have been thinking about how one might design one of these reports from a bottom-up perspective. Here's the process, in a nutshell:
I have no idea.
Don't paint me in the moron category just yet. I knew there was a better UI. I had the image in my head, but I just got flustered with all of the "stuff" that needed to happen in order for that design to work. There's a lot of "stuff" that needs to be configured, configuration options that "should" be available to the user. To be more explicit, for each layer (cohort->indicator->dataset->report) there's some "wiring" that needs to happen in order for a report parameter (say "location") to be passed down through the hierarchy of objects to allow a value like "Rwinkwavu" to be used by the indicator or cohort definition when it's time to evaluate the report.
The thing that Mike and I figured out was that we really don't need to let the user configure everything when designing the report from scratch. At least not at the beginning. Let's give them just enough configurability (add an indicator) to get off the ground. We can make some assumptions for them (you want the "location" parameter to be . Then, once things are flowing, we can let them dig down and get dirty with the report configuration. So here's the mockup I came up with from those conversations:
Now, I know this isn't revolutionary ... but it's hard to make the kind of assumptions we need to make to hide the complexity from the user. And there's still the issue of designing a somewhat intuitive interface for a user that wants to dig down into the details of each indicator in order to configure, say the "location" parameter to be hard-coded. But we'll get to that feature once there's a need for it. For now, the "edit indicator" feature will have to be left unimplemented. Sorry Mike - I know that wasn't the conclusion we came to, but it feels like the right thing to do, rather than adding one more confusing element to this page. This design feels right.
Let me know what you guys think.
So this morning, instead of banging my head against the wall for another day, I decided to have a discussion with one of OpenMRS's most gifted developers. No, not Ben Wolfe ... although, there's no doubt that he is a programming genius. No the conversation with with Mike Seaton. I asked him to give me a ring to discuss these issues. He's been working on reporting for our legacy Haiti system for years and knows a thing or two about indicators and indicator reports. I was expecting to have a quick conversation about the following mockup:
But as part of a larger process (creating an indicator report), this UI is just one little piece of confusion pie. At first glance (in my mind), the entire UI for creating the indicator report looks like a messy hierarchical chaining type thingamajig because I have been thinking about how one might design one of these reports from a bottom-up perspective. Here's the process, in a nutshell:
- User creates the cohort definition (Gender Cohort Definition with "gender" parameter).
- User creates the indicator (# Male Patients with "gender" set to "M").
- User creates the dataset definition and adds the indicator to it (My Indicator Dataset)
- User creates the report and adds the dataset definition (My Indicator Report).
I have no idea.
Don't paint me in the moron category just yet. I knew there was a better UI. I had the image in my head, but I just got flustered with all of the "stuff" that needed to happen in order for that design to work. There's a lot of "stuff" that needs to be configured, configuration options that "should" be available to the user. To be more explicit, for each layer (cohort->indicator->dataset->report) there's some "wiring" that needs to happen in order for a report parameter (say "location") to be passed down through the hierarchy of objects to allow a value like "Rwinkwavu" to be used by the indicator or cohort definition when it's time to evaluate the report.
The thing that Mike and I figured out was that we really don't need to let the user configure everything when designing the report from scratch. At least not at the beginning. Let's give them just enough configurability (add an indicator) to get off the ground.
Now, I know this isn't revolutionary ... but it's hard to make the kind of assumptions we need to make to hide the complexity from the user. And there's still the issue of designing a somewhat intuitive interface for a user that wants to dig down into the details of each indicator in order to configure, say the "location" parameter to be hard-coded. But we'll get to that feature once there's a need for it. For now, the "edit indicator" feature will have to be left unimplemented. Sorry Mike - I know that wasn't the conclusion we came to, but it feels like the right thing to do, rather than adding one more confusing element to this page. This design feels right.
Let me know what you guys think.
Friday, May 22, 2009
You know it's time to backup your hard-drive ...
... when it starts to make audible "clicking" sounds every 10 seconds.
So, I figure, this is a good time for a backup and for me to finally document the procedures I'm using to clone and backup my hard-drive ... (1) to archive them for posterity (2) to make this easier the next time I need to do a backup and (3) to save some others from having to spend time figuring it out on their own.
So I have three backup options to choose from:
(1) Clonezilla: device-device disk/partition to disk/partition
Disk to disk image using Clonezilla. Since this is a manual process I only do this every month or so. However, this is the thing that will save you when your hard-drive crashes and you need to get back up and running as quickly as possibly. Cloning to a drive with the same specs as your working drive will allow you to swap out the broken drive and swap in the cloned drive without losing a beat. I just realized that I need a third internal drive to write the disk-to-disk image in order to keep one disk that will always work (i.e. what happens if both drives end up in the drink during the backup process).
Here's the process for running a Clonezilla disk-to-disk image (screenshots would be nice, but this is all done via CD so I'm not sure if it's possible without a camera)
- Insert Clonezilla CD
- Reboot System
- Answer the following questions
- clonezilla> Which language do you prefer? English
- clonezilla> Policy for handling keymaps: Don't touch the keymap
- clonezilla> Choose the mode: Start Clonezilla
- clonezilla> Choose the mode: device-device disk/partition to disk/partition
- clonezilla> Choose the mode: disk_to_local_disk
- clonezilla> Choose the local disk as source: sda 200GB Hitachi ...
- clonezilla> Choose the local disk as target: sdb 100GB Storage
- clonezilla> Set advanced parameters: -g-auto Reinstall grub to target harddisk boot sector
- clonezilla> Set advanced parameters: Use the partition table from source disk
- And finally ... answer the following paranoid, "just watching your back" questions ... I love this part.
- clonezilla> Are you sure you want to continue? y
- clonezilla> Are you sure you want to continue? If you go on, the original data exist in the target device will be destroyed!!! Be careful!! Are you sure? y
- clonezilla> Do you want to clone the boot loader to sdb? y
- clonezilla> Now we will start to clone data to target machine ... Are you sure you want to continue? y
- Now go get some coffee or a beer (i.e. Bell's Two Hearted Ale or Dogfish Head 60 Minute IPA)
- Relax as your life gets cloned over to another hard-drive.
(2) device-image disk/partition to/from image
This is another option and one that I will likely incorporate soon, just so that I always have an extra image lying around in case something goes terribly wrong with the entire universe and this one external drive is the only thing that is spared. For this option, you basically just create the same image as the disk-to-disk image, but copy it to an external hard-drive (any old backup drive), rather than having to buy an exact replica of your current hard drive. This option just gives you a bit more piece of mind and might save you a few bucks in the short-term (you don't have to buy that second internal drive). But once your current drive goes bad (and it will), you'll need to go out and buy that second drive anyway. This just means that your recovery process will take a little longer (i.e. the amount of time it takes amazon to ship you that new drive).
I want to use all three options (just to have an image lying around in case ... yes ... something goes terribly wrong with the universe). However I still haven't adopted this approach (probably because my 500GB external hard-drive is full of Acronis' crappy backups. I hate Acronis, in case I've never mentioned it before. I'll explain in a different post, but essentially it comes down to "four months", which is the time that elapsed between when I sent in an email request to their tech support and when I received a response. If that had happened once, I would have given them a pass. But it's happened twice and I still haven't heard back from them after my last response (sometime in February 2009). For the record, Acronis is a company I want to love (they are from Burlington, MA) ... but I just can't. And until they get back to me with an apology for their terrible tech support, I will continue to give them mean looks when I pass them in the hallway.
(3) Daily backups with Ubuntu sbackup
Daily backup using sbackup (which I just found out was a Google Summer of Code project from 2005). This is going to be the next weapon I add to the backup arsenal, right after I finish my most recent disk-to-disk image.
Conclusion
So I need to get a new external hard-drive. Does anyone have any recommendations regarding backups? I'm thinking about going with a deal from Amazon that includes:
- Western Digital My Book 1TB USB
- Western Digital My Passport 320 GB USB
And does anyone out there make backups of their backups? I'm thinking about getting an external drive with a few TB of space in order to provide some redundancy in case one of the backup drives fail. So would love some suggestions on that front too.
Friday, May 8, 2009
Update: OpenMRS Test-a-Thon (May 16 - 17)
What the heck is the OpenMRS Test-a-Thon?
The OpenMRS Test-a-Thon is a code-a-thon event that allows people to collaboratively increase and improve our testing process in order to make the OpenMRS platform a more stable and robust piece of software (click here for more details).
When is it going down?
We'll be starting at noon (EDT) on Saturday, May 16 and ending at noon (EDT) on Sunday, May 17. You can come and go as you wish. I'll be hanging out in our IRC Channel all day Saturday and will be taking in a feature film on Saturday night (9 - 11pm CDT). So no pressure. Come inside, submit yourself a junit test, and then go back out and finish your Saturday chores.
Where is the event going to take place?
The test-a-thon will take place online. You can join us on the OpenMRS IRC Channel. Contact me via Skype (jmiranda) or on the IRC channel to discuss how you might be able to help out.
Who can participate?
Anyone. Developers, implementers, supporters, documenters ... people who are bored.
Who should participate?
Everyone.
Why should you participate?
Because ...
You can head over to the OpenMRS wiki for more details.
Thank you,
The OpenMRS Team
The OpenMRS Test-a-Thon is a code-a-thon event that allows people to collaboratively increase and improve our testing process in order to make the OpenMRS platform a more stable and robust piece of software (click here for more details).
When is it going down?
We'll be starting at noon (EDT) on Saturday, May 16 and ending at noon (EDT) on Sunday, May 17. You can come and go as you wish. I'll be hanging out in our IRC Channel all day Saturday and will be taking in a feature film on Saturday night (9 - 11pm CDT). So no pressure. Come inside, submit yourself a junit test, and then go back out and finish your Saturday chores.
Where is the event going to take place?
The test-a-thon will take place online. You can join us on the OpenMRS IRC Channel. Contact me via Skype (jmiranda) or on the IRC channel to discuss how you might be able to help out.
Who can participate?
Anyone. Developers, implementers, supporters, documenters ... people who are bored.
Who should participate?
Everyone.
Why should you participate?
Because ...
- You love OpenMRS.
- You want to learn about test driven develement (TDD).
- You want to learn about behavior driven development (BDD).
- You want to learn about cool unit testing frameworks (i.e. JUnit).
- You want to learn about cool web testing frameworks (Selenium).
- You want to learn about continuous integration.
- You want to write documentation for OpenMRS.
- You want to meet the team.
- You want to make Ben Wolfe happy.
- You want to win a prize.
- You have nothing better to do on a Saturday.
You can head over to the OpenMRS wiki for more details.
Thank you,
The OpenMRS Team
Warning: DO NOT INSTALL Subclipse 1.6 for Eclipse 3.4.x
The Problem
Save yourself a few days of headache ... don't do it! More details to come.
So Mike (mseaton) and I recently upgraded to Eclipse 3.4.2 because of some messed up issues with JSP/XML Validation within Eclipse 3.4. We also both installed Subclipse 1.6 thinking it would bring us joy. Three days later, with lots of back-and-forth collaboration on a project that we have sitting in Subversion, we've experienced nothing but headaches. When he's committed code to the repository, I'll update my working copy only to find that the only changes that come through are ones that are non-conflicting. So when I then try to commit my changes, I get warnings that my working copy is not up-to-date. I think the problem might be the fact that the new "SVN Update to HEAD" feature does not do a --force (but that's just a hunch). I just finished reverting my Eclipse upgrade to Subclipse 1.6 and have successfully reinstalled Subclipse 1.4.7.
Now the issue is that Subclipse 1.6 (which uses Subversion 1.6 wc format) has upgraded all of my working copy metadata to Subversion 1.6. Installing Subclipse 1.4.7 (which uses Subversion 1.4 wc format) means that I cannot use Subclipse on my currently checked out code. Since I have about 30+ projects checked out, this is kind of a problem. I can't waste hours
Here's the error you'll get when
So thankfully, the subversion folks have a downgrade script:
Download the script and run the following command:
Now, at least, I can run svn commands on that directory.
However, when I open Eclipse I can no longer see the very helpful icon overlays on top of files that are "unchanged" or "dirty". I just get unadorned, naked file icons.
Anyone have any ideas on how to fix it?
"Taste my sad, Michael" -G.O.B Bluth
Update:
So the "icon overlay" was looking more and more like an Eclipse issue, so I decided to start Eclipse with the "-clean" option.
That fixed it. And still a few hours short of what it would have taken to reinstall Eclipse, plugins, and re-download all source code.
"Taste the happy, Michael" -G.O.B Bluth
Update:
It's still broken. Only the top-level folders were successfully restored after the -clean restart. After a few restarts and rebuilds, none of the files or folders have the icon overlay. So we're back to square one.
Following Ben's advice, I tried to install Subversive, but this did not help either. This seems to be a problem with Eclipse. It's as if Eclipse is blocking the icon overlay feature of any plugin (except the core plugins that mark files with "build errors". Checking the properties of any of the files definitely shows that they are "managed" and should have at least the "managed" icon.
One last thing (check the error logs again) and then I'm going to re-install Eclipse.
Last Update:
Ok, so some combination of the above instructions, as well as closing all projects and dealing with them one at a time, while drinking coffee, and threatening to reinstall Eclipse while shaking your fist at your computer, on top of the Celtics winning Game 2 of the Eastern Conference Semifinals has seemed to fix the problem.
I can't explain it folks. Just keep playing around until it works. The one project at a time helped me focus on what I was doing.
(1) Run the script to convert from svn 1.6 to svn 1.5
(2) Clean project
(3) Build project
(4) If that fails to work, close Eclipse and restart with the (-clean) option.
The last thing I was going to try was to create a brand new workspace and move one project over at a time (running the svn convert script). It didn't come to that thankfully.
As I mentioned earlier, the last resort option would have been to reinstall Eclipse and download all of my project source code. Given that half of my day was spent troubleshooting this, I would recommend going that route when you're confronted with this issue and can't get it resolved within an hour.
Save yourself a few days of headache ... don't do it! More details to come.
So Mike (mseaton) and I recently upgraded to Eclipse 3.4.2 because of some messed up issues with JSP/XML Validation within Eclipse 3.4. We also both installed Subclipse 1.6 thinking it would bring us joy. Three days later, with lots of back-and-forth collaboration on a project that we have sitting in Subversion, we've experienced nothing but headaches. When he's committed code to the repository, I'll update my working copy only to find that the only changes that come through are ones that are non-conflicting. So when I then try to commit my changes, I get warnings that my working copy is not up-to-date. I think the problem might be the fact that the new "SVN Update to HEAD" feature does not do a --force (but that's just a hunch). I just finished reverting my Eclipse upgrade to Subclipse 1.6 and have successfully reinstalled Subclipse 1.4.7.
Now the issue is that Subclipse 1.6 (which uses Subversion 1.6 wc format) has upgraded all of my working copy metadata to Subversion 1.6. Installing Subclipse 1.4.7 (which uses Subversion 1.4 wc format) means that I cannot use Subclipse on my currently checked out code. Since I have about 30+ projects checked out, this is kind of a problem. I can't waste hours
Here's the error you'll get when
root@jmiranda-laptop:~/Workspace/module-birtreport# svn diff
svn: This client is too old to work with working copy '.'. You need
to get a newer Subversion client, or to downgrade this working copy.
See http://subversion.tigris.org/faq.html#working-copy-format-change
for details.
So thankfully, the subversion folks have a downgrade script:
http://subversion.tigris.org/faq.html#working-copy-format-change
Download the script and run the following command:
root@jmiranda-laptop:~/Workspace# python change-svn-wc-format.py module-birtreport --force --verbose 1.5
Converted WC at 'module-birtreport' into format 9 for Subversion 1.5
Now, at least, I can run svn commands on that directory.
root@jmiranda-laptop:~/Workspace/module-birtreport# svn diff
Index: .project
===================================================================
--- .project (revision 7566)
+++ .project (working copy)
@@ -1,18 +1,18 @@
...
However, when I open Eclipse I can no longer see the very helpful icon overlays on top of files that are "unchanged" or "dirty". I just get unadorned, naked file icons.
Anyone have any ideas on how to fix it?
"Taste my sad, Michael" -G.O.B Bluth
Update:
So the "icon overlay" was looking more and more like an Eclipse issue, so I decided to start Eclipse with the "-clean" option.
/usr/lib/eclipse3.4.2/eclipse -clean
That fixed it. And still a few hours short of what it would have taken to reinstall Eclipse, plugins, and re-download all source code.
"Taste the happy, Michael" -G.O.B Bluth
Update:
It's still broken. Only the top-level folders were successfully restored after the -clean restart. After a few restarts and rebuilds, none of the files or folders have the icon overlay. So we're back to square one.
Following Ben's advice, I tried to install Subversive, but this did not help either. This seems to be a problem with Eclipse. It's as if Eclipse is blocking the icon overlay feature of any plugin (except the core plugins that mark files with "build errors". Checking the properties of any of the files definitely shows that they are "managed" and should have at least the "managed" icon.
One last thing (check the error logs again) and then I'm going to re-install Eclipse.
Last Update:
Ok, so some combination of the above instructions, as well as closing all projects and dealing with them one at a time, while drinking coffee, and threatening to reinstall Eclipse while shaking your fist at your computer, on top of the Celtics winning Game 2 of the Eastern Conference Semifinals has seemed to fix the problem.
I can't explain it folks. Just keep playing around until it works. The one project at a time helped me focus on what I was doing.
(1) Run the script to convert from svn 1.6 to svn 1.5
(2) Clean project
(3) Build project
(4) If that fails to work, close Eclipse and restart with the (-clean) option.
The last thing I was going to try was to create a brand new workspace and move one project over at a time (running the svn convert script). It didn't come to that thankfully.
As I mentioned earlier, the last resort option would have been to reinstall Eclipse and download all of my project source code. Given that half of my day was spent troubleshooting this, I would recommend going that route when you're confronted with this issue and can't get it resolved within an hour.
Friday, May 1, 2009
Walk MS 2009: Part 2
This is my mom (the one whose head is being covered by my giant hand). Her name is Claudia Angel Miranda. She's 56 years old. And she has Multiple Sclerosis.Her middle/maiden name is no accident -- she had to be an angel to put up with the crap that we put her through when we were kids. But aside from possibly being an angel, she is also one of the greatest moms I've ever known. Heck, she's one of the greatest people I've ever known. Her strength, kindness, sensitivity made me the person I am today. I love the heck out of this woman.
My mom started having symptoms of Multiple Sclerosis in her mid-20's. She was diagnosed with MS around her 30th birthday. Multiple Sclerosis is one of the more confusing and misunderstood diseases out there. You need an M.D. just to understand the Wikipedia article about MS. But here are some facts:
There is no cure.
There are no known causes.
It physically and mentally debilitates people.
It affects each person in different ways.
It is painful to cope with MS.
It is painful to watch someone you love cope with MS.
This is a difficult subject for me - I’ve never been comfortable talking about my mother's condition. And it's never been easy for me to ask for help. However, I've realized over the years that my silence is not helping and that it’s extremely important for me to begin talking about MS in order to make more people aware of the disease.
Last year, with the help of friends, family and co-workers, I raised $3000 (ranking in the top 100 for the Wisconsin Chapter of the MS Society).
Thank you to everyone who donated last year.
This year, Megan and I are hoping to raise $5000. Yes, I realize that's quite a bit of dough, but $3000 seemed like a lot last year and we did it!
We don't need large donations. We just need lots of small donations. So donate $1, $5, $10, $25. It all adds up.
OR
Join us for the MS Walk in Madison
Thank you so much for reading.
Friday, April 17, 2009
Walk MS 2009: Part 1
In a few weeks (Sunday, May 3rd 2009), Megan and I are going to be participating in the 2009 MS Walk in Madison, WI. I’m blogging to ask you all for a donation (of any size) to help in the fight against Multiple Sclerosis. Last year, with your help, we raised: $2500.00. Not bad for our first year.$5000.00
So please help us achieve this goal by making a tax-deductible donation to the National Multiple Sclerosis Society! Any amount will help. Give $1, $5, $10, $20. Every dollar counts.
Thank you for your support,
Justin and Megan
Some useful links:
Friday, April 10, 2009
On MySQL Being Confused
So I've been using MySQL Query Browser and Administration Tools for years now and I've never been able to figure this one out. I have a user ('openmrs') that was created within the Administration tool and granted all privileges to the 'openmrs' schema on all possible hosts ('openmrs'@'%', 'openmrs'@'localhost', 'openmrs'@'127.0.0.1'). I have verified that the 'openmrs' user can connect to the 'openmrs' schema via the MySQL command-line client and the Query Browser. Yet, the 'openmrs' user used to connect to the database via JDBC within the OpenMRS webapp cannot connect. What gives? I keep getting the following error:
I'm guessing there might be a mismatch between the mysql.db and mysql.user table that is causing the confusion. I have a workaround, but I really want to know what's going on inside MySQL's brain. For those interested, here's the simple workaround.
Attempted reconnect 3 times. Giving up.
Could not connect to database using url 'jdbc:mysql://localhost:3306/openmrs?autoReconnect=true&sessionVariables=storage_engine=InnoDB&useUnicode=true&characterEncoding=UTF-8', username 'openmrs', and pw '*******'. Connection properties can be set with runtime property: 'connection.username', 'connection.password', and 'connection.url'
I'm guessing there might be a mismatch between the mysql.db and mysql.user table that is causing the confusion. I have a workaround, but I really want to know what's going on inside MySQL's brain. For those interested, here's the simple workaround.
grant all on openmrs.* to openmrs;
Saturday, March 28, 2009
On the Dataset Builder
I wasn't sure how to incorporate these mockups into the Dataset Builder Project page on the OpenMRS.org wiki, so I've uploaded them to scribd.com and posted them to my blog. At some point, I'll work on getting these converted to Balsamiq, but for now they will just have to remain as scanned paper mockups.
NOTE:This is just a rough draft. I'll clean this up when I get a chance. Oh and it would be great if someone could tell me how to embed HTML into mediawiki.
Dataset Viewer - Dataset Examples
This is an early version of the Dataset Builder (circa 2008). Not too many features. You add some columns, choose a cohort and ... voila ... you get a dataset to play around with. There are some other features like moving/deleting columns, but the idea was to keep things simple.
Dataset Builder - Create a Dataset (v1.0)
The next version of the Create Dataset feature ...
Dataset Builder - Create a Dataset (v1.1)
NOTE:This is just a rough draft. I'll clean this up when I get a chance. Oh and it would be great if someone could tell me how to embed HTML into mediawiki.
Create a Patient Dataset
Dataset Builder - Create a Patient DatasetCreate an Encounter Dataset (v1.1)
Dataset Builder - Create an Encounter Dataset (v1.1)Create an Observation Dataset
Dataset Builder - Create an Obs Dataset (v1.0)Create an Encounter Dataset
Dataset Builder - Create an Encounter Dataset (v1.0)Examples of Datasets
Dataset Viewer - Dataset Examples
Create a Dataset (v1.0)
This is an early version of the Dataset Builder (circa 2008). Not too many features. You add some columns, choose a cohort and ... voila ... you get a dataset to play around with. There are some other features like moving/deleting columns, but the idea was to keep things simple.
Dataset Builder - Create a Dataset (v1.0)
Create a Dataset (v1.1)
The next version of the Create Dataset feature ...
Dataset Builder - Create a Dataset (v1.1)
Thursday, March 26, 2009
On the Quartz Scheduler Project
I have been trying (in vain) to get the Quartz Scheduler project page up on the OpenMRS wiki, but keep getting the following error when posting my changes:
So for now, the project page will just have to be left here.
There are many activities within a healthcare information system which benefit from automation. For example: daily clinic reports might need to be generated the morning before a clinic opening, decision support rules might need to be run over a patient group once every week, or healthcare data coming in from outside sources might need to be analyzed every 30 mins. To give end implementations the flexibility to define these sorts of automated conveniences, the OpenMRS community has added scheduler functionality to the core software application. The OpenMRS scheduler (based on the JDK Timer) currently supports executing tasks on regular basis (i.e. a task will execute every X seconds, minutes, hours, days). However, there's no way to schedule a task to recur at a particular time every week. The data model supports this type of feature (start time and repeat interval), but we need to add some logic to the Scheduler to handle triggering these tasks to occur at exact times.
We want to support more complex scheduling (i.e. cron-like) in the very near future, where we can specify rules that will allow Tasks to run more more/less frequently based on the day/time. For instance, an implementation might want to schedule a Task to run once every day at midnight, but also at 6pm on Saturday and Sunday. This type of scheduling would be difficult to support using the JDK Timer, so we would like to add an extension point to the scheduler to allow a developer (probably myself) to create a module based on the [http://www.opensymphony.com/quartz Quartz Scheduler]. This will allow us to handle more advanced scheduling requirements.
Student will be responsible for soliciting requirements (user stories) from users/implementers. Here are just a few examples of user stories that might be applicable to this project.
Planning Process
So for now, the project page will just have to be left here.
Overview
There are many activities within a healthcare information system which benefit from automation. For example: daily clinic reports might need to be generated the morning before a clinic opening, decision support rules might need to be run over a patient group once every week, or healthcare data coming in from outside sources might need to be analyzed every 30 mins. To give end implementations the flexibility to define these sorts of automated conveniences, the OpenMRS community has added scheduler functionality to the core software application. The OpenMRS scheduler (based on the JDK Timer) currently supports executing tasks on regular basis (i.e. a task will execute every X seconds, minutes, hours, days). However, there's no way to schedule a task to recur at a particular time every week. The data model supports this type of feature (start time and repeat interval), but we need to add some logic to the Scheduler to handle triggering these tasks to occur at exact times.
We want to support more complex scheduling (i.e. cron-like) in the very near future, where we can specify rules that will allow Tasks to run more more/less frequently based on the day/time. For instance, an implementation might want to schedule a Task to run once every day at midnight, but also at 6pm on Saturday and Sunday. This type of scheduling would be difficult to support using the JDK Timer, so we would like to add an extension point to the scheduler to allow a developer (probably myself) to create a module based on the [http://www.opensymphony.com/quartz Quartz Scheduler]. This will allow us to handle more advanced scheduling requirements.
Requirements
High Level Goals
- Decide whether to build as a module to extend current functionality or as a core component to replace the current functionality.
- Decide whether to include Quartz tables in the OpenMRS data model (there are about 7 Quartz-specific tables).
- Implement a Quartz version of the scheduler to extend (possibly replace) the current JDK Timer implementation.
- Implement features that allow a user to manage scheduled tasks.
- Redesign the Manage Scheduler user interface to provide a more user-friendly and intuitive experience.
- Create unit tests for each expected behavior of every method within the scheduler service and its domain objects.
User Stories
Student will be responsible for soliciting requirements (user stories) from users/implementers. Here are just a few examples of user stories that might be applicable to this project.
- Allow a user to view all available tasks.
- Allow a user to view all scheduled tasks.
- Allow a user to create a scheduled task.
- Allow a user to update a scheduled task.
- Allow a user to delete a scheduled task.
- Allow a user to schedule tasks to be run on an interval (i.e. every 30 seconds) or at specific moments in time (Tue, March 24 at 10:42pm).
- Allow a user to schedule emails to be sent at a specific time.
- Allow a user to schedule reports to be generated and sent to stakeholders at a specific time.
- Allow a user to start an available task.
- Allow a user to stop a scheduled task.
- Allow a user to check whether a task is currently executing.
Planning Process
- Identify user roles that will be using the scheduler
- Identify user stories for each role by interviewing users / stakeholders
- Prioritize user stories using MoSCoW rating (Must, Should, Could, Want)
- Estimate user stories (a.k.a Planning Poker) using ideal days (0.5, 1, 2, 3, 5, 8, 13, 20)
- Discuss acceptance criteria (expected behavior) for each user story with stakeholders.
- Create a Release backlog for the current sprint / iteration
- Estimate / calculate velocity (i.e. 20 story points)
- Create an Iteration backlog of stories
- Create tasks for each user story in the iteration
- Assign estimated hours to each task
- Mockup user stories prior to start of sprint to solicit feedback.
- Run sprint for iteration duration (between one to two weeks)
- Deploy features completed during sprint to labs.openmrs.org
- Solicit feedback from users/stakeholders
- Re-prioritize and re-estimate users stories / tasks
- Recalculate velocity
- Create an Iteration backlog for stories that fit into
- After X iterations, deliver final release
Timeline
This section will be completed by the mentor and student.- Iteration #1:
- Iteration #2:
- Iteration #3:
- Iteration #4:
- Iteration #5:
- Iteration #6:
- Release:
Mockups
This section will be completed by the student.FAQs
Still compiling questions/answers. Will post by Friday.Resources
Project Resources
- [http://docs.google.com Project Discussion] (has not been created yet)
- [http://docs.google.com Release Backlog] (has not been created yet)
- [http://docs.google.com Iteration Backlog] (has not been created yet)
OpenMRS Resources
- [http://openmrs.org/wiki/Scheduler_API Scheduler API]
- [http://openmrs.org/wiki/Quartz_Scheduler_Developer_Guide Quartz Scheduler Developer's Guide]
- [http://openmrs.org/wiki/Quartz_Scheduler_User_Guide Quartz Scheduler User Guide]
- [http://openmrs.org/wiki/Technical_Discussion#Task_Scheduling Technical Discussion about existing Scheduler]
- [[ticket:357|Ticket #357]]
- [[ticket:359|Ticket #359]]
- [[ticket:1056|Ticket #1056]]
Blogs
- [http://blog.justinmiranda.com Justin Miranda]
- [http://www.openmrs.org Student Blog]
External
- For more information on Quartz, please see the [http://www.opensymphony.com/quartz Quartz Project] home page.
- more resources to follow
Thursday, March 5, 2009
On Transparency and Staying On Task
The Problem
If you've ever wondered, "What the heck is Justin doing right now?" ... you're not alone. I often wonder what it is I do every day.
The Tangent
Sorry, this reminds me of a scene from Office Space, that I would be remiss to miss quoting:
The Problem (again)
Most of the time, a day passes and I am absolutely exhausted, I feel like my mind hasn't stopped running all day and yet I don't feel like I've actually accomplished anything. I know that I have ... just probably not what I was supposed to be doing. Well, I'm tired of that feeling. Over the years, I've tried every possible system to help me with time/project management: simple ToDo lists, GTD, Agile, Agile + GTD, GTD + FYI + BTW, and even Tony Robbin's tapes. I've worked with systems like Jira/Trac to keep track of the larger tickets/projects, GTD with Day-Timers, GTW with hipster PDAs (moleskin), GTD with real PDAs (BlackBerry 8830) to keep track of the daily tasks, and SlimTimer (which is awesome, by the way) to keep track of actual time spent on tasks. I've used online todo lists with RememberTheMilk, simple todo lists on $1 notepads, electronic todo lists on $200 BlackBerries or in Thunderbird Tasks, or even in Notepad (ala Yaw Anokwa, who also incidentally gave us Bacon-wrapped OpenMRS).
Nothing seemed to work.
And I'm not entirely sure why these technologies and processes haven't helped, but I think it might to do with the fact that I have too many inputs: email, e-task list, paper-based task list, jira, trac, skype, meetings). Another problem is that I'm always "too busy" to follow up on the tasks I've worked on or completed, to leave notes about where I was or to close tickets, so my inbox just keeps growing and growing and the contexts for those tasks keep getting lost. Still not sure how to handle that problem other than to schedule some time every week for "cleanup" and assessment. Unfortunately, the time gets scheduled away with something else. How do others deal with that issue? I guess it's just a matter of will-power and stopping yourself when it's time to move onto the next thing.
The Solution (Staying on Task):
Anyway, I've come up with an approximation of a solution to the problem. I recently decided to try to integrate all of these various task/time/project/life management tools into something that will help me stay on task and keep everyone informed of what it is that I'm up to. It involves integration of a few key systems that each, by themselves, help solve a little piece of the puzzle and together, I'm hoping for better throughput and transparency all around.
Storing Tasks
I love Remember the Milk, but I hate logging into a web site to manage what I should be working on. Ideally, I'd rather have someone (or maybe something) tell me what I should be doing at any given time during the day. Unfortunately, I still haven't found a solution to that. However, RTM at least gives me a way to manage my todos, a database for all of my tasks. Which is a good first step. I've now got an @OpenMRS context in RTM with all of my daily todos from some of my in-progress and future OpenMRS/PIH tickets.
Managing Tasks
Now, I just need something to help me manage tasks, that doesn't require me to log into a web site several times a day. RTM has a great feature that allows you to sync with the BlackBerry (as well as iPhone). I am now using that service to manage tasks when I'm away from my computer. Unfortunately, it doesn't work so well when I'm sitting in front of my computer because it takes a few more seconds to type in the task using the BlackBerry's tiny keyboard. And seconds are hours in the software world (or maybe just in my world). So, I needed something that was sitting right there in front of me, on my laptop, whenever I needed it. I tried using the Thunderbird RTM Provider that integrates with RTM, but ran into a few bugs that were showstoppers. The add-on definitely has promise and I would highly recommend it once it matures for a few more weeks/months.
That brought me to Tasque, which is a neat Gnome project (for Ubuntu) that was developed during the 2008 Google Summer of Code. Tasque integrates with the RTM backend and allows me to add/remove/edit tasks from my desktop. There are still some quirks in Tasque (it won't let me move tasks to different contexts once they've been added), but the pro's far outweigh the con's.
Managing Time Spent
The final problem to tackle was that of "time spent" on tasks. This one was easy. Since, I always have at least one FireFox window open at all times, I use SlimTimer in the sidebar of Firefox to keep track of the time that I spend on individual tasks. I haven't perfected the system yet, but I current track high-level task categories, like Code Review, Design, Development, Testing, Meeting, and Troubleshooting. I will probably get more fine-grained at some point. Unfortunately, there's currently no way to integrate SlimTimer with RTM, so I have to add tasks to the SlimTimer sidebar manually. Keeping the tasks as high-level categories gives me the consistency I need to actually use SlimTimer every day.
The Solution (to the other problem ... "Transparency")
The other nicety from all of these systems working together is that I now I have a way to show what I'm working on all in one place. You can see some of the feeds I've set up for open tasks/tickets, as well as completed tasks/tickets (coming soon). Hopefully that will provide some needed transparency and help others give me feedback on what they think I should be working on.
The Conclusion
So in conclusion, what I settled on was a combination of Jira/Trac to manage big tickets, RTM + Tasque + BlackBerry Tasks to manage daily todos, and SlimTimer to manage the time spent on those tasks. We'll see how this new system goes.
If you've ever wondered, "What the heck is Justin doing right now?" ... you're not alone. I often wonder what it is I do every day.
The Tangent
Sorry, this reminds me of a scene from Office Space, that I would be remiss to miss quoting:
Bob Slydell: What would you say ... ya do here?
Tom Smykowski: Well look, I already told you! I deal with the g*dd*mn customers so the engineers don't have to! I have people skills! I am good at dealing with people! Can't you understand that? What the hell is wrong with you people?
The Problem (again)
Most of the time, a day passes and I am absolutely exhausted, I feel like my mind hasn't stopped running all day and yet I don't feel like I've actually accomplished anything. I know that I have ... just probably not what I was supposed to be doing. Well, I'm tired of that feeling. Over the years, I've tried every possible system to help me with time/project management: simple ToDo lists, GTD, Agile, Agile + GTD, GTD + FYI + BTW, and even Tony Robbin's tapes. I've worked with systems like Jira/Trac to keep track of the larger tickets/projects, GTD with Day-Timers, GTW with hipster PDAs (moleskin), GTD with real PDAs (BlackBerry 8830) to keep track of the daily tasks, and SlimTimer (which is awesome, by the way) to keep track of actual time spent on tasks. I've used online todo lists with RememberTheMilk, simple todo lists on $1 notepads, electronic todo lists on $200 BlackBerries or in Thunderbird Tasks, or even in Notepad (ala Yaw Anokwa, who also incidentally gave us Bacon-wrapped OpenMRS).
Nothing seemed to work.
And I'm not entirely sure why these technologies and processes haven't helped, but I think it might to do with the fact that I have too many inputs: email, e-task list, paper-based task list, jira, trac, skype, meetings). Another problem is that I'm always "too busy" to follow up on the tasks I've worked on or completed, to leave notes about where I was or to close tickets, so my inbox just keeps growing and growing and the contexts for those tasks keep getting lost. Still not sure how to handle that problem other than to schedule some time every week for "cleanup" and assessment. Unfortunately, the time gets scheduled away with something else. How do others deal with that issue? I guess it's just a matter of will-power and stopping yourself when it's time to move onto the next thing.
The Solution (Staying on Task):
Anyway, I've come up with an approximation of a solution to the problem. I recently decided to try to integrate all of these various task/time/project/life management tools into something that will help me stay on task and keep everyone informed of what it is that I'm up to. It involves integration of a few key systems that each, by themselves, help solve a little piece of the puzzle and together, I'm hoping for better throughput and transparency all around.
Storing Tasks
I love Remember the Milk, but I hate logging into a web site to manage what I should be working on. Ideally, I'd rather have someone (or maybe something) tell me what I should be doing at any given time during the day. Unfortunately, I still haven't found a solution to that. However, RTM at least gives me a way to manage my todos, a database for all of my tasks. Which is a good first step. I've now got an @OpenMRS context in RTM with all of my daily todos from some of my in-progress and future OpenMRS/PIH tickets.
Managing Tasks
Now, I just need something to help me manage tasks, that doesn't require me to log into a web site several times a day. RTM has a great feature that allows you to sync with the BlackBerry (as well as iPhone). I am now using that service to manage tasks when I'm away from my computer. Unfortunately, it doesn't work so well when I'm sitting in front of my computer because it takes a few more seconds to type in the task using the BlackBerry's tiny keyboard. And seconds are hours in the software world (or maybe just in my world). So, I needed something that was sitting right there in front of me, on my laptop, whenever I needed it. I tried using the Thunderbird RTM Provider that integrates with RTM, but ran into a few bugs that were showstoppers. The add-on definitely has promise and I would highly recommend it once it matures for a few more weeks/months.
That brought me to Tasque, which is a neat Gnome project (for Ubuntu) that was developed during the 2008 Google Summer of Code. Tasque integrates with the RTM backend and allows me to add/remove/edit tasks from my desktop. There are still some quirks in Tasque (it won't let me move tasks to different contexts once they've been added), but the pro's far outweigh the con's.
Managing Time Spent
The final problem to tackle was that of "time spent" on tasks. This one was easy. Since, I always have at least one FireFox window open at all times, I use SlimTimer in the sidebar of Firefox to keep track of the time that I spend on individual tasks. I haven't perfected the system yet, but I current track high-level task categories, like Code Review, Design, Development, Testing, Meeting, and Troubleshooting. I will probably get more fine-grained at some point. Unfortunately, there's currently no way to integrate SlimTimer with RTM, so I have to add tasks to the SlimTimer sidebar manually. Keeping the tasks as high-level categories gives me the consistency I need to actually use SlimTimer every day.
The Solution (to the other problem ... "Transparency")
The other nicety from all of these systems working together is that I now I have a way to show what I'm working on all in one place. You can see some of the feeds I've set up for open tasks/tickets, as well as completed tasks/tickets (coming soon). Hopefully that will provide some needed transparency and help others give me feedback on what they think I should be working on.
The Conclusion
So in conclusion, what I settled on was a combination of Jira/Trac to manage big tickets, RTM + Tasque + BlackBerry Tasks to manage daily todos, and SlimTimer to manage the time spent on those tasks. We'll see how this new system goes.
Wednesday, March 4, 2009
On Features and Feedback
A few months ago, at an AspirationTech conference in Oakland, CA (one the best conferences I've ever been to - by the way), I found out about UserVoice ... a website that provides "users" [of a product] a chance to voice their opinions about what features that'd like or bugs they want fixed.
So, if you're an OpenMRS user and you have an idea for a feature, click on the "Feedback" link on the left edge of your browser to submit an idea to OpenMRS. This is kind of an experiment at the moment, but I'm hoping we can solicit some good suggestions from our users and give you guys "a seat at the table" as we start planning out our road map for the upcoming (already in-progress) year.
There's also GetSatisfaction. I haven't used it yet, but have heard some good reviews. If you have comments about either or another feedback tool, please let us know.
So, if you're an OpenMRS user and you have an idea for a feature, click on the "Feedback" link on the left edge of your browser to submit an idea to OpenMRS. This is kind of an experiment at the moment, but I'm hoping we can solicit some good suggestions from our users and give you guys "a seat at the table" as we start planning out our road map for the upcoming (already in-progress) year.
There's also GetSatisfaction. I haven't used it yet, but have heard some good reviews. If you have comments about either or another feedback tool, please let us know.
Thursday, January 29, 2009
On Cloning a Hard-Drive: The Problem
Over the past three years I have had the worst luck with computers: 2 system board crashes due to unknown issues, one system board crash due to an accident involving some water and an ethernet cable, two or three hard-drive failures, wireless drivers that mysteriously stop working and won't allow you to uninstall/reinstall.
The common theme to all of these issues, is that I've almost always lost at least a weeks worth of work trying to get back up to speed, re-installing the OS and software, configuring a development environment, downloading email, etc.
Needless to say, I either need to stop using computers or figure out a backup/cloning strategy.
For now, I chose the latter.
I just finished setting up Ubuntu Intrepid 8.10 on my laptop. I installed Eclipse, MySQL, Tomcat, Java, downloaded email, blah blah blah ...
So now, I want to clone this hard-drive so I can swap in the cloned drive if the first drive ever fails. I will clone the primary drive onto the second drive once a week (or more often if I can find some software to manage it for me).
I currently use Acronis True Image Home 11 for my backups, but for some reason it does not allow me to clone the drive. After contacting Acronis "support" a few times, I have decided to start looking for an open-source solution to the problem.
The common theme to all of these issues, is that I've almost always lost at least a weeks worth of work trying to get back up to speed, re-installing the OS and software, configuring a development environment, downloading email, etc.
Needless to say, I either need to stop using computers or figure out a backup/cloning strategy.
For now, I chose the latter.
I just finished setting up Ubuntu Intrepid 8.10 on my laptop. I installed Eclipse, MySQL, Tomcat, Java, downloaded email, blah blah blah ...
So now, I want to clone this hard-drive so I can swap in the cloned drive if the first drive ever fails. I will clone the primary drive onto the second drive once a week (or more often if I can find some software to manage it for me).
I currently use Acronis True Image Home 11 for my backups, but for some reason it does not allow me to clone the drive. After contacting Acronis "support" a few times, I have decided to start looking for an open-source solution to the problem.
On Cloning a Hard-Drive: The Solution
Thanks to Renaud and Jim for their input.
Jim - At some point I'll adopt a "constant backup" solution (using an online service or local backup server), but for the time being, I think I found a winner.
So I want to clone the hard-drive (source) that I've spent the past few days configuring onto a second drive (target) so that I can swap the two drives without worrying about reinstalling applications, reconfiguring email accounts, etc. The idea is that I will clone the source drive onto the target drive at least once a week so that I can stay up-to-date with Ubuntu patches and packages. And if something goes wrong with the source drive, I can plug the target drive in and I'll be ready to go. I'm also planning to make and copy an .iso of the source drive over to another external hard-drive just in case things don't always work with the disk-to-disk cloning.
In order to solve this problem, I used CloneZilla 1.2.1-23 to clone my current Ubuntu setup (with all applications, emails, and other data) over to a second drive (target).
First of all, the size of the target drive matters -- it cannot be smaller than the source drive. That's pretty obvious, but I was hoping there would be some way for CloneZilla to see that only a portion of the source drive was being used and copy just that data over to the target drive, resizing partitions as needed.
No big deal. I just had to play around with the partitions on both drives to make sure that the source drive was smaller than the target drive. There are plenty of tools that allow you to edit the partitions on any of your attached hard disks, but it's not very safe to edit the partition used to boot into Ubuntu since the source drive is mounted.
Therefore, I created a new bootable CD off the latest System Rescue CD ISO (http://www.sysresccd.org) in order to run gparted from outside Ubuntu. This CD is a life saver and has a lot of great tools. Resizing the partitions took about 10 minutes and once that was done, the CloneZilla part was a breeze.
Next time I run through the CloneZilla process I'll write the exact instructions down. For now, I'll just say that I walked through the CloneZilla wizard and gave logical answers to questions, answered "yes" to all of the paranoid "are you sure?" questions, and after about 20 minutes, the target drive was cloned and bootable (CloneZilla also installs GRUB on the new drive).
Jim - At some point I'll adopt a "constant backup" solution (using an online service or local backup server), but for the time being, I think I found a winner.
So I want to clone the hard-drive (source) that I've spent the past few days configuring onto a second drive (target) so that I can swap the two drives without worrying about reinstalling applications, reconfiguring email accounts, etc. The idea is that I will clone the source drive onto the target drive at least once a week so that I can stay up-to-date with Ubuntu patches and packages. And if something goes wrong with the source drive, I can plug the target drive in and I'll be ready to go. I'm also planning to make and copy an .iso of the source drive over to another external hard-drive just in case things don't always work with the disk-to-disk cloning.
In order to solve this problem, I used CloneZilla 1.2.1-23 to clone my current Ubuntu setup (with all applications, emails, and other data) over to a second drive (target).
First of all, the size of the target drive matters -- it cannot be smaller than the source drive. That's pretty obvious, but I was hoping there would be some way for CloneZilla to see that only a portion of the source drive was being used and copy just that data over to the target drive, resizing partitions as needed.
No big deal. I just had to play around with the partitions on both drives to make sure that the source drive was smaller than the target drive. There are plenty of tools that allow you to edit the partitions on any of your attached hard disks, but it's not very safe to edit the partition used to boot into Ubuntu since the source drive is mounted.
Therefore, I created a new bootable CD off the latest System Rescue CD ISO (http://www.sysresccd.org) in order to run gparted from outside Ubuntu. This CD is a life saver and has a lot of great tools. Resizing the partitions took about 10 minutes and once that was done, the CloneZilla part was a breeze.
Next time I run through the CloneZilla process I'll write the exact instructions down. For now, I'll just say that I walked through the CloneZilla wizard and gave logical answers to questions, answered "yes" to all of the paranoid "are you sure?" questions, and after about 20 minutes, the target drive was cloned and bootable (CloneZilla also installs GRUB on the new drive).
Subscribe to:
Posts (Atom)