Thursday, December 17, 2009

On backing up and restoring large mysql databases

In order to improve the performance of mysql when dealing with large databases, there's a couple of settings that can be tweaked in your my.cnf file (or through a MySQL administration tool). The following line has proven to be the most useful.

innodb_buffer_pool_size = 1G

I've been meaning to write about this improvement for the last few weeks, but I wanted to test it myself first. So far I'm very pleased with the performance improvement (database restore time is down from about 3 hours to about 45 minutes on a ~1.2G database). I haven't tested how this setting affects MySQL during normal execution, but it certainly helps improve performance during backups and restores. So if you find yourself backing up and restoring large MySQL databases, you should test this out. You should (obviously) not set innodb_buffer_pool_size to a value greater than the available memory on your machine.

Thanks to Marc Harrison for the tip.

If you want to read some more about this configuration, I found a couple of blog entries from a few years ago Enjoy!

http://www.mysqlperformanceblog.com/2007/11/01/innodb-performance-optimization-basics/
http://www.mysqlperformanceblog.com/2007/11/03/choosing-innodb_buffer_pool_size/

Wednesday, December 16, 2009

On setting up a development environment in Mac OS X 10.6 Snow Leopard

I just bought the new Mac Mini from Apple. It's my first Mac since the Mac Plus of the early 90's and I have to say ... it feels good to be back. I still love doing development on Ubuntu, but Mac OS X is easy to get used to and a lot of fun to play around with. Nothing compares to "apt-get install ..." (on Ubuntu) when getting a development environment set up, but this tutorial for Tomcat 6 probably saved me a couple of hours of agony of going it alone.

Thursday, December 3, 2009

Fixing an existing subversion project to migrate to the trunk/branches/tags convention

Here's the procedure I used to move an existing "dumb" subversion project to one that follows the subversion convention (trunk/branches/svn). It's a combination of the two ideas presented in the articles below. More steps then I was hoping for, but it's a pretty simple-to-follow solution and much less aggravating than the Eclipse approach.

(this procedure assumes that you are using Linux, but the process would be similar on Windows or Mac)

// 1. Checkout a fresh copy of your project
$ svn co http://svn.myproject.com/myproject myproject

// 2. Change directories to make it easier perform svn operations
$ cd myproject

// 3. Add the new trunk directory
$ svn mkdir trunk

// 4. Locate all files/dirs in the current directory and svn move them to the trunk directory
$ find . -maxdepth 1 ! -name trunk -exec svn mv '{}' trunk \;

// 5. Double check that all files have been svn move'd (.classpath, .project, build.xml, etc)
$ ls -la trunk

// 6. Commit all changes
$ svn ci -m "myproject: moving all files from myproject into myproject/trunk"

// 7. Switch your local copy of the module to
$ svn switch http://svn.myproject.com/myproject/trunk myproject


Resources:

Thursday, October 29, 2009

OpenMRS Developers Conference (Oct 2009)

OpenMRS Developers Conference in Latin America

The following document was used by the developers/trainers at the OpenMRS Developer's Conference in Lima, Peru. We kept track of the agenda and topics we discussed during the week in order to possible create a curriculum for future training sessions. I'd be glad to discuss this with anyone who is looking to create training manuals for both short and long-term training sessions on OpenMRS-related technologies.

This document is still pretty raw, but feel free to use bits and pieces if you find them useful.
http://docs.google.com/View?id=dnc9s5q_10gb5qb4dn

Monday, September 21, 2009

Extending the OpenMRS Reporting API: Cohort Queries

Introduction
At the Cape Town meeting last week, we demo'd (if you could call it that) the new reporting framework and some of the early versions of tools that we've built on top of it. Some of those tools have been described in recent entries and others are still undocumented. That will change soon. The UI and tool set for the reporting framework is still immature, but we're happy with what we've come up with from an API standpoint. As a developer, you *should* be able to do most of what you need with what exists. However, there will be instances where you need to extend the framework to do what you want.

NOTE: If anyone knows a good plugin to include code snippets in Blogger, please let me know.

Extending the Cohort Query
Here's a quick example of how to use the API to evaluate a cohort definition. This will return you a list of patients that can be used as input into another piece of code.

// Define the cohort definition
GenderCohortDefinition gcd = new GenderCohortDefinition();
gcd.setGender("M");

// Evaluate the cohort definition
Cohort males = Context.getService(CohortDefinitionService.class).evaluate(gcd, new EvaluationContext());


This is a pretty trivial example, but there are lots of cohort definitions in the system that will allow you to do more complex queries. With that said, there will likely be a time when you need to implement a cohort query that is not currently supported within the framework.

For instance, let's say that the Ministry of Health within the country you work calls your mobile and says "Hey, Justin ... we need to report on the number of patients that have had the XYZ form completed on a certain date or in the past month/year".

In order to extend the system to support this query, we need to do three things: (1) write a new cohort definition to define this query, (2) write the evaluator to evaluate the cohort definition and return a cohort.

0) Write a JUnit test that tests what your new cohort definition should do



public class FormCohortDefinitionEvaluatorTest extends BaseModuleContextSensitiveTest {

private Log log = LogFactory.getLog(this.getClass());

@Test
public void evaluate_shouldReturnPatientsWhoHadFormCompletedOnDate() throws Exception {
FormCohortDefinition fcd = new FormCohortDefinition();
fcd.setForms(Context.getFormService().getAllForms());
Calendar newYears = Calendar.getInstance();
newYears.set(2009, 0, 1);
EvaluationContext evalContext = new EvaluationContext();
evalContext.addParameterValue("sinceDate", newYears.getTime());
evalContext.addParameterValue("untilDate", newYears.getTime());

Cohort cohort = Context.getService(CohortDefinitionService.class).evaluate(fcd, evalContext);
Assert.assertEquals(1, cohort.getSize());
}
}


1) Write the cohort definition
So the first step is to define the cohort definition. We just need to extend the DateRangeCohortDefinition to allow dates to be specified as properties or parameters and add a "forms" property that will be used to allow a user to select forms in the system.


public class FormCohortDefinition extends DateRangeCohortDefinition {

private static final long serialVersionUID = 1L;

@ConfigurationProperty(required=false)
private List<Form> forms;

public FormCohortDefinition() {
super();
}

public List<Form> getForms() {
return forms;
}

public void setForms(List<Form> forms) {
this.forms = forms;
}
}


2) Write the cohort definition evaluator

@Handler(supports={FormCohortDefinition.class})
public class FormCohortDefinitionEvaluator implements CohortDefinitionEvaluator {

/**
* Default Constructor
*/
public FormCohortDefinitionEvaluator() {}

/**
* @see CohortDefinitionEvaluator#evaluateCohort(CohortDefinition, EvaluationContext)
*/
public Cohort evaluate(CohortDefinition cohortDefinition, EvaluationContext context) {
FormCohortDefinition fcd = (FormCohortDefinition) cohortDefinition;
Date fromDate = fcd.getCalculatedFromDate(context);
Date toDate = fcd.getCalculatedToDate(context);
Cohort cohort = new Cohort();

PatientSetService pss = Context.getPatientSetService();
for (Form form : fcd.getForms()) {
List types = null;
Cohort tempCohort = pss.getPatientsHavingEncounters(types, null, form, fromDate, toDate, null, null);
cohort = Cohort.union(cohort, tempCohort);
}
return cohort;
}
}


I want to use the API (instead of writing my own DAO, so we have to do something slightly stupid here. The PatientSetService.getPatientsHavingEncounters(...) method only takes a single form as a parameter, therefore I need to iterate over the forms in the cohort definition and call the API each time. If performance was an issue (and if I had more time) I would write a DAO that does the actually query I want. But neither of those statements are true, so this is a perfectly good solution.

3.) Now, let's can go back and run our JUnit test ... and hopefully see green.

INFO - LoggingAdvice.invoke(102) |2009-09-21 17:13:46,399| In method CohortDefinitionService.evaluate. Arguments: CohortDefinition=null, EvaluationContext=org.openmrs.module.evaluation.EvaluationContext@1d742a1,
INFO - LoggingAdvice.invoke(127) |2009-09-21 17:13:46,528| Exiting method evaluate
INFO - FormCohortDefinitionEvaluatorTest.evaluate_shouldEvaluate(51) |2009-09-21 17:13:46,536| # of patients: 1



Friday, August 7, 2009

On the Cohort Dashboard (implementation)

So here's the first pass at an implementation of the Cohort Dashboard. The only thing that is currently working is the fact that we can pass this page a cohort. The rest of the functionality still needs to be implemented. Let me know what you think.


Thursday, August 6, 2009

On the Cohort Dashboard (design)

This is the page that will complement the Cohort Builder. The page will be accessible by users who want to analyze a set of patients from the top-down. More details to come ...