College of Engineering and UK ITS Collaborate to Recover Data

UK College of Engineering and UK Information Technology Services (UK ITS) worked together over the 2022 winter break to quickly recover data on equipment damaged from a burst pipe. On December 25th, 2022, a pipe in the mechanical room of Anderson Tower burst due to cold temperatures in Kentucky. This burst caused water to pour through a hole in the ceiling, directly onto the equipment in the College of Engineering server room. Unfortunately, due to the University being closed, the damage was not discovered until the following day. 

In total, approximately 10 servers were lost to water damage,   The impact was severe, causing significant data loss, as well as rendering many of the  College of Engineering’s critical servers unusable. 

Kyle Dippery, IT Manager in the College of Engineering, reflected on the days following the incident, saying water was still draining on the 27th -- two full days after the pipe burst.  While waiting on the waterlogged servers to dry, Dippery and his team were able to repurpose other equipment to have  a working helpdesk, two working file servers, and two license servers up by December 30th.  

After restoring basic functions, the College of Engineering called UK ITS for assistance.  Ray Hyatt, System Operations team lead in UK ITS, started working right away to restore the data that was lost. Due to a couple hurdles restoring the data, the process took multiple days with several members of UK ITS and College of Engineering IT working together to resolve the problem.  Teamwork between the College of Engineering and ITS was instrumental in overcoming the disaster. Recovery efforts continued until January 11, 2023, until UKITS was finally able to recover the lost data thanks to UK ITS’s Data Protection Service. Hyatt described collaboration with the College of Engineering during this incident as “...friendly, professional, and knowledgeable of their services. They kept their cool in a very stressful time. This is not always the case as any worker in IT will tell you.” 

Recommendations from Hyatt and Dippery for keeping your data safe and secure in the event of a failure or emergency:

1.  Make plans beforehand: Think about single points of failure-- power supply, fans, hard drives, network interfaces, CPUs, motherboards, room cooling, water pipes, even buildings and campuses in extreme events. Have a backup plan for when any of the aforementioned items fails or is unavailable to you regardless of the reason.
2. Trust no one thing with your data: If your data is worth storing on a computer, it is most likely worth backing up.Your time is valuable, so don’t potentially waste it by having to recreate something lost on a lost hard drive. 
3. 3-2-1 rule: 3 Copies of your data, 2 different media types, 1 in a remote location. Why? A USB stick in your desk drawer in the same room where your workstation is burns in the very same fire, disappears in the same theft, blows away in the same tornado, OR washes away in the same flood.