01/06/2020
Launching new data management systems often means working on large data clean up in preparation for a data migration project.
We would all prefer data clean up to be an easy and automated task. Excel has some useful tools to help spot exact duplicates, but if there’s an extra comma or period, most of us have to rely on some old fashioned, roll up your sleeves, manual data clean-up.
Here are some tips from some very experienced data clean up staff at IPLS:-
1. Break it down into sections or stages. For example, stage one is name clean up. Stage two, focus on the dates. Block off hours in the day to get it done if you have the time over several days.
2. Demand no other distractions – no other job responsibilities. You don’t answer your phone or your emails or participate in office chit chat. If you have your own office put up a “Do Not Disturb” sign. Open plan office? Put on headphones and a baseball cap to signal to others you are working on a project. If you can’t avoid other work – work on the project in the early mornings before anyone else arrives.
3. Take frequent breaks – rest your eyes frequently, every 10-15 minutes. Staring at a screen is very hard on the eyes.
4. Adjust lighting on your screen and around you – low/no overhead lights and darken your screen. If you can’t adjust overhead lighting, that baseball cap can help block out the overhead light glare.
5. Don’t let your mind wander – as soon as you are not focusing on the data, stand up, take a break, then start again. Certain music types help some of us focus. Some of us swear by EDM (Electronic Dance Music) while others prefer classical or even head banging rock music for focus. Find what works for you (and don’t judge the others in their choice of music).
6. Save your work often – every 10 to 15 minutes – or more frequently.
7. In a crunch? There’s going to come a time when you can do no more. Have a 2nd person ready to take over so the work continues and you hit your deadline.
Remember, clean data means no duplications, no extra commas, no missing commas, no extra spaces and, often, no foreign characters. Common pitfalls are date formations – make sure every entry has the same date formation – e..g, dd/mm/yyyy.
Look for typos. Sort and re-sort the data often looking for duplicate names, client numbers and account numbers.
Understand any input values and what they mean. Data migration may require input of certain values in certain columns. For example, “True” or “False”. Often it’s not clear what the values mean. Make sure to ask if you’re not certain.
Your data is valuable – keep it clean and make sure all staff that enter data into your systems are aware of how critical accuracy in data entry is. Good luck!