Transforming a legacy system: It's 1980 something

The year was 1980 something and somebody decides they need a computer system to collect and track revenue.  Sweet!  Let's grab the IBM luggable, hit the turbo button and write some kick ass code.  Don't forget your floppy... or make that two, we're not sure how much code we'll be writing.

But wait a second, what's the best tool for the job?  Fortran and COBOL are out there; they're nice proven tools, but are sooo 1970.  How about Basic?  Nah, Basic was for kids who liked stuff like this:

10 PRINT "POOP"
20 GOTO 10

... that was actually my first program on my Commodore.

C would be a great choice, but the tool is a bit loosy goosy and I'd hate to overflow the super secret double linked global buffer when I assign my help string to a 2 byte word by mistake.

How about Pascal?  Now we're talking.  A mature language and those guys over at Borland are doing some nice stuff with their Turbo line of products.  So you pick up a few copies of Turbo Pascal version something.something and go to town writing some POS code.  You sell some stuff, you collect some payments, you record some revenue, you report on some revenue... now you're cooking with gas.

Let's take a step back and wrap some context around this stuff.  It's 1980 something.  Xerox had just started playing around with the concept of a graphical interface, Apple had already stolen the idea for the Lisa and Macintosh, and Windows 1.0 just came out.  Personal computers were just starting to take off and breaking speed limits @ 8 megahertz.  The typical IBM PC shipped with MS-DOS, had a whopping 640k of memory, dual floppy drives, a 1200 baud modem and networking was a mostly foreign concept.

Now back to reality - you've got to build software to run on this pig.  To that end, here's a quick tangent.  I'm not that old of a guy, but am old enough that I was around to remember this stuff and to write code in this environment.  I truly believe that folks learning about software development today missed out on the golden age of our craft.  I doubt a kid coming out of college these days will relate well to developing software to run on 640k of memory - in a world when consuming too much memory didn't mean that Windows would dig into the magical abyss of virtual memory.  Instead, we'd get the super awesome heap overflow runtime error and your application would stop dead.  I could come up with 50 similar examples of stuff we had to worry about that are concerns of the past today, thanks largely to riding the wave of Moore's Law.

Here's some stuff you did back in the 80's when you built large systems.  Some of this stuff you did because it was the best way to do it.  Some of this stuff you did because it was the only way to do it.

Global variables
Most versions of Turbo Pascal didn't support objects - without objects, encapsulation was a bit tricky.  Knowing what I know today I could probably fudge something that resembles encapsulation using a procedural development model, but who'd think about that in 1985?  After all, most people didn't know about unit testing and global variables were so darn convenient.  So you need some variables to keep in memory so that sub-systems can communicate state.  You're smart, so you create a couple source files and group logical storage containers together so that it makes sense.  This also gives you some semblance of cohesion in sub-systems.  Good stuff.  A wise man taught me this back in the mid 90's.  You're doing alright if you can build your systems with high cohesion and low coupling.

End result - a bunch of source files, with related global variables.

Pointers
Due to memory limitations of the hardware running this software, you had to worry about memory consumption.  Specifically, you thought about stuff like stack and heap utilization.

Remember all those global variables we just talked about?  They consume stack space.  We didn't have a ton of stack space back then and if you consumed too much, you got a big fat stack overflow runtime error.

Another slight tangent.  You had to worry a lot more about writing efficient code when running on an IBM 8088 or 286 CPU with a snail of a 10 megabyte hard disk.  God forbid you forgot to turn on write caching in SmartDrv.  You couldn't constantly hit disk for data or your system would run like crap.  You had to strike the perfect balance between the consumption of conventional memory and hitting disk too often.

So you stored stuff in memory so that you didn't have to hit disk too often.  This stuff in memory consumed stack space.  At some point you ran out of stack and got stack overflow errors.  To combat the lack of stack space you start moving stuff to the heap - that magic area of memory used for dynamic storage allocation.  How was this accomplished?  You started converting your large footprint global variables to pointers so that stack consumption was minimized and these global pointers on the stack pointed to stuff in the heap.  If you ran out of heap you were pretty much out of luck.  Try rebooting.  If that didn't work it was time to get back to the drawing board (or maybe you could tweak out your CONFIG.SYS to load some stuff in the upper memory blocks).  

End result - a bunch of pointers that pointed to stuff.  If you were lucky you had global variables that pointed to records with fields that were three dimensional arrays of records that pointed to more stuff... because that's how we rolled.

Nested Sub-Routines
I mentioned earlier that encapsulation was a bit tricky before the shift in paradigm to objects.  One clever technique that was used was to nest routines inside of other routines.  If you had a really complex routine you may nest many routines in support of it.  In one way this was super fancy because you were able to hide that nested logic from other callers.  This also had the unfortunate side affect of increasing the scope of any local symbols in the mother routine.  Today we know this is a bad idea and with private class members we can avoid such temptations.  We also know to limit the scope of a given symbol as much as possible.  Passing data to a private method is much preferred over using private members to communicate data.  The trend towards unit testing has driven us further from large classes with many private methods to smaller classes, sometimes with only a method or two, which lessens the temptation to be lazy with our class members.

End result - mega sub-routines with many, many nested routines, all with access to the main set of local symbols.

Entropy
The universal laws of entropy very much apply to software.  In the biz, we call it "code rot".  Code rot is the concept that code will become more and more disorderly over time if not properly tended to.  I will state up front here that our system is generally cared for.  It's the life blood of the company and we take our software very seriously.  Having said that, I think it's natural that software written in the 80's and 90's will naturally degrade over time as platforms change, development paradigms shift and your development team turns over.  

When we made the jump from our 16-bit compiler to our 32-bit Windows compiler we decided to not start over from scratch.  We had too much invested in the software to start over.  The benefit of this approach was that we were able to jump to a Pascal based Windows compiler and take advantage of a million lines of useful code.  It's hard to measure how much this helped our time to market with our Windows product.  On the flip side, this decision held back the rate in which we were able to modernize our large and aging source base.  To be clear, large and aging doesn't mean faulty.  It means we've got stuff like global variables, pointers and nested sub-routines that we have to build into classes at some point.

End result - an older code base that's large and a bit coupled.  

Where to go from here
Throughout my years working on this system, I've always been involved with our core POS sub-system, but have never been directly responsible for it.  This recently changed - I was challenged with modernizing this core part of our system so that we can take full advantage of what the last 15 years of software evolution offers us.  Classes, encapsulation, polymorphism, dependency injection with IoC, unit testing, interface abstraction, plug-able architectures, etc.  I'm up for the challenge -  read on if you want to follow the process.

Pleased to meet you

Hey there, I'm Ron.  I'm a software developer, a father and a husband.  I'm one of the lucky ones out there who loves what they do.  Building software happens to be what I do for a living, but it's also a hobby of mine.  I've been developing since the days of my Commodore 64 so I think I have a few things to say about development that might be worth reading about.

In my day job, I work for a software shop that builds a very large scale POS revenue collection system.  Our system runs worldwide with tens of thousands of end users and billions of dollars flow though our system  annually.  Does that sound big?  I hope so, because it's a huge system.  The company has a rich history and equally rich software.  The system was born in the 80s and has evolved ever since.  New features are conceived of and build constantly.  As our industry and customers evolve, we evolve; to stay on the cutting edge with what we do and to aid our customers in building their businesses.

I was fortunate enough to join the company in the early days (1996 to be exact) so I've seen the software evolve and was round for many of the major milestones of the system.  Watching such a large system evolve over the years has taught me lessons that you'll never learn in school.  I've been though development methodology changes, compiler changes, IDE changes, platform changes, product changes, source control changes - the list goes on and on.  If there's a mistake to be made, we've made it.  If there were things to learn, we've learned them.  In my field, you never stop learning and I hope to chronicle a bit of that knowledge here.