Yarn, Tar Balls and Reindeer

I had a dream, nightmare really.  One of those compulsive things, like the Face Peeling scene from Poltergeist, where you know you should stop but whatever you're doing is more important than the risk you're taking.

I was making something out of a ball of yarn or twine.  At first, progress was good.  The yarn came off easily and went right where is was suppose to. But as I got further, the yarn got bits of tar clinging to it that stuck to my fingers and progress slowed.  The deeper I got into the yarn, the thicker the tar and my hands became covered and sticky.  But stopping wasn't an option and there was no other yarn about.  Eventually, hands and arms covered in tar, touching anything damaged the work I had previously finished.  Finally, approaching a point of despair, I looked up and out of the shadows walked a reindeer with a fresh ball of yarn."

This, of course, was not really a dream.  Rather, it's an analogy for what we've experienced with the USxS Redesign projects.   Software developers use frameworks, developed by someone else, to build their software on.  For web development, there are hundreds of tools to choose from, each with their own strengths and weaknesses.  Selecting the correct tools is one of the more important decisions I have to make as a project leader.  These tools have a steep learning curve and represent a significant investment in time and resources. They also represent a significant risk if you select the wrong one.

A few years ago, we began using a web development framework called Apache Tapestry. Tapestry is a good tool and we've successfully used it in other projects.  At first, we had good success and made good progress. However, the USxS-R data models are very "dynamic" (e.g.. the district can add their own fields).  This causes a problem for most web frameworks, including Tapestry, that are "template" based.  The tools, generally, want you to declare exactly where every field goes on the page.   We managed to teach Tapestry to be more dynamic to a point.  But the further we went, the more we struggled at making the applications work the way we wanted and our productivity suffered.  

When you estimate software projects, you have to get two main things right: scope (how much work there is) and productivity (how much you can do in a unit of effort).  Our early efforts with Tapestry led me to make estimates that were wrong given the later results.

"It's a poor craftsman that blames his tools".  Tapestry is a fine tool, but it was not the right fit for the applications we tried to write with it.  You can argue that I made a bad choice choosing Tapestry, and with the advantage of hindsight, you may be right.  But I certainly made a mistake in failing to recognize the core problem sooner.  It's a hard thing to switch frameworks in the middle of an already delayed project after investing significant effort and resources. 

The SSDT and I have a thirty year track record of successful software delivery.  This particular error is the only significant blot on my record.

Nevertheless, over the winter, it became clear we couldn't continue.  So we picked up a new tool called Vaadin (a Finnish word meaning female reindeer).  After testing and prototyping, we committed to Vaadin in the spring and began integrating and writing (and re-writing) interfaces.  Vaadin is a very different type of framework.  Being based on GWT (originally Google Web Toolkit), it excels in creating high quality, interactive user interfaces with less effort. It's "template-less" so the the page components are generated dynamically.  Vaadin is a much better fit for the USxS applications.   We still have much work to do to, but we've accomplished more in six months with Vaadin than previous year and a half with Tapestry. 


Before I go further, there's a simple demo available at:


The demo contains sample data that does not represent a real district (the numbers won't make sense).  I suggest you play with the Vendor option (and perhaps compare it to the "Legacy Vendor" option).  But feel free to play with any part of it you like. Be sure to try the "filtering" in the Vendor grid (that is, type a name under the "Primary Name" column).  It's cool.

If you're at all technically minded, noticed that the URL points to my personal hosting account (on DigitalOcean).  This is a true cloud deployment running entirely outside of any ITC.  It's running as two Linux containers (one for the webapp and another for the SQL database). It's costing me $0.03 per hour to host this instance.  If someone buys me a beer at OEDSA this week, it will pay for a week of hosting... 


I'm loath to do this, since most of my previous schedules and estimates have been crap, but I know you expect it, so here it goes.

To date, we have been issuing "Milestone" releases hoping to get feedback, primarily, from ITC personnel.  Milestones were released when we reached a significant feature point in our backlog (remaining work).  We may, or may not, issue additional Milestone releases.  But the remaining back log has been reduced enough that we now need more feed back directly from school districts.

Therefore, by the end of (calendar year) 2015, we will be issuing "Preview Releases".  A "Preview Release" is more than a "Milestone" but less than a "Release Candidate".    The goal of the preview releases will be to solicit feedback directly from school district personnel to help us prioritize the remainding backlog.  

We intend to provide deployable releases to the ITCs to allow them to host test instances (see below).  For ITCs not  capable of hosting, the SSDT may provide temporary hosting directly.  Either way, the district's real data will be loaded into the tests instances, to allow full evaluation and testing by districts.

Assuming good results from preview testing, we should be ready for release candidates and production releases in the Spring of 2016.


A word about "conversion".  If you've ever converted to another accounting or payroll system, you know the typical conversion happens on a fiscal or calendar year boundary.  This is because the new system only imports "master" data and the transaction data is not converted.  With most conversions, your not really converting, you're dumping your old data and starting fresh.

This will not be the case when upgrading from Classic USxS to Redesign.  The new applications will import 100% of your transactional data and will post GL and encumbrance journals, create posting periods, et al. automatically.  We will load all the history that is in your current system.  Therefore, it's quite likely that you'll be able to upgrade in the middle of a month and everything will balance.  (We'll be testing this during preview releases).

Technical Bits

If you're not technical, the next part may not interest you.  

As a quick review, the Redesign projects are based on a "single-tenant" architecture.  Meaning that each school district will have at least two servers (per application).  The first being a JavaEE (Tomcat) application server running Java 8.  The second, a database server running Postgresql.  A district running both USAS and USPS will have four servers (perhaps three if we consolidate the databases).

I can admit to you now, that at the time I committed to this architecture, I was terrified of the possibility of having to manage 1600+ virtual machines state-wide.  Back then, "cloud computing" was clearly the hot thing, but 1600 servers is still 1600 servers!  But it did not take too much imagination to realize that the IT world would have to invent better ways to manage large cloud deployments.  So I placed a bet on "single tenant" and assumed the universe would solve the problem.

I'm happy to report that this gamble paid off.  While we were busy writing the software, the world invented "containers".  The best known form of this is Docker.   I won't go into the details here, but the primary advantage is that containers allow you to run multiple/many containers on a single VM without the overhead of reproducing the OS and libraries. The demo running at http://demo.cloud.davesmith.me/ is a instance of USAS-R web application plus a database server deployed  as two Docker containers.

 The SSDT plans on providing deployments (Docker images) of the Redesign to be hosted by ITC's or elsewhere.


If you lasted to the end and attending OEDSA this week, see me with any questions, or email me, or post on the Answers site.  If you buy me a beer, I'll keep the demo running an extra week. :)

p.s. When this post hits Google, a Tapestry developer or two will almost certainly want to flame me.  So I've disable commenting here.  You're welcome to flame in email, I might even answer.