Tuesday, May 12, 2015

Working in the financial industry

I worked at Sammons Financial Group for almost 7 years, first as the only DBA for the annuity division, later as part of the enterprise DBA team.  Besides working way too long days, this was a great opportunity to expand my knowledge of SQL.  Worked with some very talented people on both the .NET side and the SQL side, created some applications from some crazy specifications, and had 80 or so databases on one server.

Got my feet wet with the use of SSIS and SSAS, snuck in the use of SSRS while none was looking.  SSRS turned out to be highly liked by the user base, for once they could get ok looking reports fairly quickly, and they could set up their own schedules for when they wanted them and how to have them delivered.  Win.

After I had run SSRS for the Annuity division for about a year, the enterprise decided to do so as well.  They paid for a consultant to take a look at what they needed for hardware etc., without consulting me.  The specs they gave him for how many reports at the enterprise level was 100, at that time I had over 150 reports just for one division.  Fail.

During my years at Sammons I upgraded the annuity division from SQL 2000 to SQL 2005, and finally to 64 bit SQL 2008.  What a relief to be on 64 bit after dealing with too little memory for so many years.  Our production server went from 4 GB to 32, then to 64+.  Since we had a somewhat subpar SAN, keeping the caches warm for hours instead of seconds was a major plus.

Being the only DBA in charge of a division took it's toll on my life, and I decided a change was needed since one did not occur internally when they where given a chance.  The market in Des Moines is fairly small, especially if you do not want to work for any of the banks located there.  A few years earlier I had been in Chattanooga, TN on a motorcycle vacation that my wife insisted I go on.  I think she might have had ulterior motives!  So we started looking at cities within riding distance of the Blue Ridge mountains.  Soon I received an offer from AgData in Charlotte, NC.

This finally brings us to today.  Part of a team of 7 developers, 2 DBAs, working with a single client (it's a large client).  For the first time part of a team where the other DBA is competent, and we both can cover 100% for each other when the other is out.  Been MANY years since I could take a vacation for 2 weeks and leave my work behind completely with no worries about there being a meltdown waiting for me.  Even to the point that I went to Norway over Christmas 2014 in the middle of a large project, a project I was responsible for moving to production a week after coming back.  Very nice!

So here we are, up to date with where I am at.  Hopefully found a city we want to live in for a long time.  Weather is nice, not as hot as Texas was, nor as cold as Iowa was.  Yet we have 4 seasons.

Here is the reason why I love living in this area, best motorcycle roads pretty much anywhere.

Now that you know a few things about me, on to talking about SQL!


Thursday, May 7, 2015

Going back to my roots, DBA at an Aviation company

Late October 1999 I was in Norway celebrating my moms 60th birthday, to her great surprise as I was not supposed to be there.  The day after her birthday party I got a call from my boss telling me I had to get to Houston ASAP.  A cancer treatment company had just had their DBA walk off the job due to not handling the stress very well.  2 week engagement until they could get things under control.  When I got there a few days later I learned that they had known Y2K issues that had not been fixed yet.  All in all I stayed there for 6 months, after fixing their Y2K issues at 6 PM on New Years Eve.  Cutting it a bit too close.  The longer I stayed the more I understood why the previous DBA had walked off the job, high stress when your servers run radiation machines in about 30 hospitals around the country.

After this I was ready to not be a consultant any more.  Took a job with FlexJet in Dallas,  being one of 3 DBAs, managing their database servers that ran their fleet management program.  FlexJet at the time had about 100 business jets that would fly wherever their customers/owners needed them, being that they were time shares we had a lot more owners than planes.  Due to this the scheduling of aircraft was extremely complex even compared to an airline, at least they know in advance what the schedule should be, we generally knew about 4 hours ahead of time where we needed a plane.  Getting the right size airplane, to the right airport, could be a challenge.  True 24 hour operations with world wide coverage (although most of it in the US, or at least US based owners).

The SQL code at FlexJet was full of code that was deprecated.  To make sure we could upgrade to newer versions of SQL I went through every line of SQL code in the company to make sure it had no deprecated features, this included rewriting stored procedures, as well as rewriting dynamic queries in C++.  Joy!  The fun thing with this job was to use some of my skills learned in flight training and operations classes, being able to speak the language on both sides has some benefits.  When I for personal reasons decided to move on (I considered moving back to Norway), they asked me to come work in their Copenhagen office and they would pay for my commute every week.  Was very tempting.

Instead, I decided it was time to get my feet wet with replication.  Something I had never done before.  On the interview, as soon as the pleasantries where over, I opened by stating "I know nothing about replication, have never done it before, but so far have not run into anything in SQL I could not figure out.  If that is not acceptable, let us not waste any time and call this interview over."  I was there from the beginning of the project until 3 months after going live and handing it over to a permanent DBA to handle.

Replication was a challenge for sure.  This was for a book seller, at the time they had about 75 stores.  We set up replication to send inventory records out to each store as books where loaded on the trucks, then they where made active when they got scanned at the receiving end.  We used about every different kind of replication possible, even to the extent that we could query sales live on the main server at head quarters, yet they could operate the stores even if communication was down.  For early 2000s this was quite a big deal, last time I checked in with the DBA there 2 years ago the system was still in use, but quite expanded both by number of stores and scope of their capabilities.  If their project has gone right, the new system should be out by now.  Except for me, I believe all the programmers involved in the first version is still there.  Great testament to the work culture at this company, one of the very few places I was on contract that I was sad to leave.

2002 came with the troubles after 9/11 and I was laid off for a few months, spent some time studying up on holes in my SQL skills while looking for a job.  Being unemployed sucks.  Finally landed a job with ABC Radio Networks (strangely enough taking over the job one of the DBAs from a previous job had held), responsible for databases running the radio operation.  Had a great team of programmers and network engineers, some of which are friends to this day.

Had some interesting challenges at ABC.  How to deal with 53 week years?  WHHHAAT?  Yeah it happens, and the way they wanted the report was 53rd week compared to the 1st week of the same year.  Quarterly reports?  Don't even ask.  I have to admit I did a design flaw here by calculating it, should just have had a look up table that defined the reporting period.  OR convinced them to use normal reporting where you would just compare week by week, and in the 53rd week the comparison year by year would be to the same week as the previous week.  Live and learn.  The function that would calculate this was one of the nastiest piece of code I have ever written.

One of the systems I took over had a table with a 12 column primary key, foreign keyed into a table with 16 columns primary key.  Surprisingly it was pretty fast.  Was a good challenge to write an archiving routine for this table, since when I got there we had 100s of millions of records in the table, most of which where not needed.  The arching job ran in batches for the better part of a month to move data off the primary file.  Later on SQL came out with partitioned tables for this.  Oh how life has gotten easier over the years.

Wrote a search application that could search their entire transcribed news archive.  In the beginning the users wanted to be able to search the result of the search, since their current search application was super slow they thought this was needed.  Luckily convinced them to let me try it my way and just run a search all over again every time they wanted to narrow down the result.  The day I demo'ed it for a few users they demanded I get it in production immediately, the searches all took less than 2 seconds on the entire data set, vs. 10-15 minutes in their current application.  SQL wins!

I was there when Citadel bought ABC Radio Networks from Disney, due to certain activities I decided to move on.  Having forgot that I had turned on nationwide searches while unemployed, the first call I got was from a company in Des Moines, Iowa.  When they offered me the job I was watching TV with my wife, I turned around and asked her if she would be ok with moving to Des Moines.  Yupp.  After over 11 years in Dallas, 2 weeks later I had moved from Dallas, TX to icy cold Iowa.  Had snow on the 5th day I was there.  This might not have been a good move.  Traffic was, however, much better.