[SCADASEC] Setting up a Control Systems Group

[Brodsky, Jake]

This is for those of you who are wondering what it's like before and
after forming a group dedicated to Control Systems.  

I originally started working at WSSC as a telecommunications technician
while going to Engineering School at night.  I saw a view of the
company's telecommunications infrastructure from the inside.  A few
years later, I graduated and was about to shop my degree around, when my
bosses decided they liked what I was doing and wanted to keep me there.


I began working on the control systems of the company.  At that time,
the systems were very rudimentary.  Almost all controls of any
significance were manual.  We were heavily staffed with operators
24/7/365.

Implementing the earlier control systems wasn't easy.  There were
algorithms to be developed, controllers to be debugged, wiring to figure
out, HMI screens to set up, and so on and so forth.  This was made all
the more difficult because we were working for different divisions in
the company.  I got to know my counterparts in other divisions pretty
well.  But at the end of the day, we'd have to get together in meetings
to try and sell our fearless leaders on what we felt they should be
doing.  We were herding our bosses to coordinate funding, set policies
that worked across divisions, and so forth.  

The company went through major reorganization in 2000.  The
reorganization results were quite mixed in my opinion.  However, one of
the things which seems to be working very well is the creation of a
"Process Control Group".  We now have a budget, we have assets to
manage, and we can coordinate comments on all design projects.  We don't
get people buying computers and software for us at the very beginning of
a three to five year construction project any more.  (This sort of thing
was commonplace prior to the group getting together)

We're basically in-house IT networking, consultants, integrators, and
technicians with a steady stream of work.  Given that we have six plants
with substantial control systems and a wide area distribution/collection
SCADA system, there will be a steady stream of work for the foreseeable
future.  The nice thing is that once the job is over, we're still
available to tweak things.  

This breeds a certain pragmatism about our work.  While many projects
specify complexity up front because that's the only time they'll get
access to the engineering/programming resources, we live with our
creations.  We don't put any more stuff in our control systems than
we'll need at startup.  If it needs to get more complicated, we can add
that in at any time.  Conversely, if something breaks in the middle of
the night, we WILL get called to fix it.  We learn from our mistakes,
because if we don't we'll get hollered at by one of our own.  Nobody
likes having to break away from home life to fix something that we
already had dealt with before.

This also removes the need to front-load a great big project with all
the conceivable control system designs that we might want over the next
five to ten years.  Thus, the validation is simpler, the software is
more straightforward, the complexity shows up only where you need it,
and not for reasons having to do with features that nobody uses.  

When it comes to security, we have a view of the whole picture, not just
a piecemeal view (job by job) of a security problem.  We have the
authority to purchase, install, and update company wide control system
security features.  We are in a much better position to evaluate what
our operators need and what they want, because we have to live with the
results too.  Once again, this is where our pragmatism for installing
only security features we need comes from.  

Oh, and did I mention that we're building large control systems for a
fraction of the installed price that most other utilities would pay?  

Our in-house customers, the Operations staff, like this way of doing
business.  We think it serves the company very well.  I hope our new
management doesn't see fit to tinker with it too much.  

Jake Brodsky