USAS/USPS Redesign Projects Status Update

At the OEDSA Fall 2010 Conference,  I presented an update to the USAS/USPS Redesign Projects.  The USAS-R project has been running since October, 2009 and the USPS-R project started in August 2010.

Attached to this post are two PDF files containing the slides from the presentation.  The first is the presentation intended for end-users and ITC Fiscal support staff.  The second PDF (-technical) is basically the same presentation but with some additional technical details for the Technical Track. The second session goes into more detail on the Architecture and tools being used to develop the new system.

For the rest of this blog post, I'll summarize some of my comments during the presentation and include a few screen shots of the demo I did of the functioning software.

Big Picture Overview

To put the Redesign projects in context, the "Redesign" project is really Phase II of the "Migration Strategy".  Phase I of the Migration Strategy, involved creating web based user interfaces for the existing (COBOL/main-frame) backend.  I won't re-hash the Migration Plan here.  For more info, see State of the USAS Address.

Phase II of the migration is to replace the 30 year old platform with a modern RDMS (SQL) using modern languages and tools.

The first and primary goal of the Redesign is to reproduce "the existing functionality of USAS".  We are not intending to "enhance" the software significantly.  However, we are "re-thinking" the data model for USAS to create a flexible model that will be able to grow in the future. The data model for the new systems (the way data is stored and related) will be much different than the current COBOL-based model.  However, the new systems must maintain a certain amount of compatibility with the Classic systems (to allow importing, backwards compatible SOAP services and compatibility with the existing USAS web application).

Please see the attached PDF's for examples of how the new data model will differ from the existing Classic USAS.

Although we are not intending to enhance the software for this stage of the project, there will be a certain number of "incidental" enhancements.  The act of moving the software to a modern platform and tools creates enhancements as a natural side-effect.  See "Custom Fields" and "Domain Events" in the PDF as examples.

Database/Server Architecture

Early on in the project, we (in consultation with the SAC), decided on a "Single District per Database" model. This is how USAS and USPS have always been structure (one set of RMS files per district).  We considered a "Multi-Tenant" model (all districts in one large database), but decided that the costs of backup/recovery, scaling and software complexity argued for a Single Tenant Model.

Going further with the Single Tenant model, we then decided to go with a single "software installation" per District.  This is a significant departure for State Software.  Traditionally, the software has been installed once at an ITC and shared by all districts at that ITC. In a "Single District Install", each district will have it's own "Virtual Server" which will be independent of other districts.  This yields a number of potential benefits:

  • Customization per District (only loading modules in use by the district)
  • "Cloud ready" (ability to run a districts server in an internal or external server cloud)
  • Easier scaling (spread load of multiple districts across physical servers)
  • Improved Security and less software complexity (programmers need to write less security related code)

Of course, a server per district will place a heavier administrative burden on ITC support staff simply due to the number of virtual servers they may be responsible for.  So this will place a bit of a burden on the SSDT to make the software more manageable then the Classic system.  For example, we will need to develop (or acquire) an "Administrative Console" and easy software upgrades.  We believe we can automate many of the current administrative functions using common off-the-shelf components.

ODBC is dead

A potential shocker in the presentation is that ODBC is likely to be of little use in the Redesign'ed models.   ODBC has been very important in Classic USAS and USPS for the last few decades.   We were able to provide ODBC access to the Classic systems because of a product called Safari UDMS.  UDMS allowed us to encode security and calculated fields into the "databases" via views and custom code.

The new data models, however, are very unlikely to have "Views" that are consumable by end-users or even technical ITC staff.   This is a little hard to explain in text (see the diagram on page 6 of the PDF), but the application is being developed at a "higher level" than the database.  The new database will be a simple data store without any business logic (e.g. calculation of totals) and without user-level security. 

In other words, if you were to access the database directly with ODBC, you would be bypassing the application security and you would have to understand the data model sufficiently to calculate values (e.g. to calculate a PO total you'd have to total all the item amounts).

To replace the functionally currently provided by ODBC, the new systems will contain a "Reporting Service". The reporting service will "flatten" and simplify the data model for reporting needs.   It will supply the same business logic (calculations) and apply the user-level security.  It will also have a fairly simple "query language" what will allow you to filter and sort by any field in the database.  Further, we will provide a full complement of output formats (PDF, HTML, Excel, CSV, tab-delimited, XML, etc).

So, although you may not access the data directly with ODBC, there will be flexible, easy way to extract the data you need from the new systems.

Below is a screen shot of a "prototype" reporting interface for a simple "Vendor Listing".  This is not a complete interface.  It is meant only to give you an idea of what a possible user interface for reporting might look like.

A few things to notice about this screen shot:

  • The default query allows filtering by Number (range) or Name. But the "Advanced Query" block can filter by other fields.
  • The "Columns" block allows you to choose which fields to include on the report.

Below is a shot of what this report might look like (in PDF).

Domain Events

One significant architectural feature of the Redesign is the idea of "Events".  Domain Events or "Application Events" is a technique for allowing modules within an application to communicate with each other without necessarily being aware of each other.  This makes the software less complex and more flexible.  See the PDF for examples of Events and Listeners.

We added Events to the system in order to help with "SIF readiness" in the future and to make some parts of the system easier to implement.  However, an interesting (incidental) side-effect of adding events is the possibility of "User Defined Events".   A user defined event might listen for certain events and react to them in different ways.

For example, perhaps the Treasurer in a school district might like to receive an email if a Purchase Order for more than $100,000 is created.   Or, perhaps in your district, an email address should be required for all Vendor records.   With Domain Events, it will be relatively easy to allow end-users (with appropriate permissions) or ITC staff to write "custom" listeners through a web interface.

In other words, it will be possible for users or ITC's to customize the way that USAS works internally for a given district.

Conclusion

The SSDT has made substantial progress on replacing the back end.  The basic framework and data model has been laid down.  The tool set we've selected is allowing us to write more functionality with less code than ever before.  There are still a  "thousand little details" to work out before we're finished, but the progress to date is promising.

We hope to begin issuing "milestone releases" in the first half of next year.  These releases will be fully packaged 'virtual servers" that can be downloaded and used to import real data from a school at an ITC. Districts who are interested can have a "test server" to try with their own data.  Then, we can start making lists of things that are missing in order to reach our goal of "existing functionality of USAS".   When the "Missing List"  is down to a reasonable number, we'll start releasing "production releases".

One difficulty of replacing a system like USAS is that "existing functionality" may mean different things to different users. It's very likely that the "Missing List" will be different for each District.   Some Districts may decide that the early production releases are sufficiently complete to use in production, and others may choose to wait.   This is probably a good thing since it will allow ITC's and the SSDT to spread the implementation cycle of a period of time.

If you have any questions or concerns, feel free to leave a comment below or drop me an email at smith@nwoca.org

AttachmentSize
oedsa-usas-r-design-and-status.pdf507.7 KB
oedsa-usas-r-technical.pdf528.95 KB