This is a series describing various architectures I have worked with in the last two decades and some of their benefits and issues. The table of contents of the series is here.
Evant was originally named Retail Aspect and provided a Retail-as-a-service suite to companies that were joining the Web retail boom (e.g. Disney Online).
Major System Aspects
The technical foundations of the company were from a Java / Smalltalk background, so the server technologies were pretty mainstream Java enterprise technologies. The client was the ‘leading edge’ piece in the implementation technology, using a lot of JavaScript back in a very early time for the language (early 2000s). The whole system was notable in the number of automated regression tests it contained (see below). The database was initially Oracle but later moved to DB2.
Evant had a suite of products that did not succeed as a suite, potentially due to 9-11 causing a shutdown of online retail activity. But the Advanced Planning system was of interest to several retailers, including Staples.
AA-1 (Architectural Aspect): Strong Client, Server-UI, and Server-Domain separation
In terms of making the Evant Advanced Planning product capable, performant, and testable, there was a very strong separation between “Interface” (UI or Test) and “Domain”. The Domain includes all the business functionality within the planning engine, exposed by a Java-based API. It could be driven by either tests or the User Interface. The API was identical, so if the tests were successful, the engine was doing the ‘right thing’. And the UI just needed to:
- Interact with the interface similarly to the tests
- Or expand interface and tests for new needs
- Present the information pleasantly and effectively
The UI could do all kinds of amazing things to transform the results or make actions easier for a user. Since this was a JavaScript application, lots of things could happen on the client without server interaction or asynchronously with the server. The important part was having a single contract that the two clients (one verifying, one using) could run against.
AA-2 : Mass Automated Testing
The original Evant team was very committed to a full XP (Extreme Programming) approach and used TDD, Paired Programming, and other aspects of XP as part of their development process. I arrived after this development period, but there were a fairly extensive collection of automated tests as part of the development artifacts. However they were created, they were incredibly useful for regression testing as we transformed the Domain to be far faster, more scalable, and flexible.
Initially the tests were in XML to allow a very flexible system of automated testing that (in theory) could have tests written by subject matter experts or general end users. This flexibility made it a poor Domain-Specific-Language and users could not write tests themselves. The tests were also very repetitive (wet) given they had to describe many states, inputs, and outputs within a matrix-like space. Ultimately the solution was to move to a matrix-oriented tool: a Spreadsheet. And simply organize states, inputs, and outputs within that spreadsheet. Automation turned the spreadsheets into automated test specifications. And the integration server ran this vast collection of tests pretty much all the time to make sure nothing regressed (or at least it was identified if it did).
The automated testing was a continuous benefit as long as we could keep performance of the testing servers equal to developer demands.
AA-3 : Hidden Storage Model
An important part of the Domain’s API was its’ separation of ‘transactions’ from its ‘storage’. The system had transactional statements (‘update’ and ‘save’) but how those things were accomplished was not visible at the interface. This separation prevented callers from caring and fiddling with how things were communicated to the persistent storage.
Not all systems need this kind of separation: What is the chance you will swap out your database? With a very different database? But the Evant storage model was a Hybrid-relational system with the bulk of the data stored in semi-opaque compressed format. So the domain acted transactionally, but under the covers it did a lot of data transformations to organize and compress facts. Transformations that evolved in time (different versions had better formats) and evolved based on the size of the data space and performance tuning around it.
AA-4 : Canned to Generic
Another common and useful architectural progression is going from ‘canned’ (fully specified) to ‘generic’ (very flexible) capabilities. You should generally start at ‘canned’ so you have super-control over what you are doing and what you expect its results to be. This is great for both modeling and testing the system. As the canned capabilities grow, they can become unwieldy and need to be more parameterized or even genericized (e.g. an Excel formula built out of operations).
As you go from canned to generic, you will likely encounter both behavioral anomalies and performance anomalies. But if you start with generics that do the same as canned, you can focus on performance. And then switch to generics that are more broadly capable and focus on whether they behave correctly. And then return to performance of these more broadly capable generics.