Is now the right time to clean up this data?

Application requirements continue to evolve over the life a project. For example, I was working on a project that allowed users to take a variety of surveys. Most users would take the survey directly with the application, but sometimes a user would fill out a paper version to be keyed in later.

Over the 2 decades that this application has been in production, the form has changed a handful of times. When questions are added or removed, the number next to each answer has to change accordingly.

But the developers that came before me, only changed the human-visible label. Internally, the old numeric labels remained unchanged. So you might see 20 on the paper/online form, but in the database, that answer was stored as 61.

Now, a new request came in to see the numeric answers the user selected on the report which was generated from a particular survey. But if I showed 61, the user would be misled because that now lines up with 20.

This made me pause and wonder: is now the time to fix these 20 years of historical data? Or do I just map the internal value to the currently-displayed value to avoid confusing users?

It would take me maybe an hour or two to do the mapping and deliver the requested feature, but it might take 2 or 3 days of effort to build and test a big migration script (there was a LOT of historical data, and multiple versions of values).

In this particular case, I decided to just map the values and deliver the feature. True, I didn't want to add days of work to a tiny change, but I also recognized that there was an element of risk in making a big data change as well.

One thing I did was to add comments to the mapping function explaining the decision. I also opened an issue in the backlog to revisit this in the future.

In a perfect world, we would want to thoroughly fix anything like this that we bump into in the codebase, but you have to be realistic and balance it with the needs of the organization using the app.

Here to help,

Joel

P.S. Did you know we offer a free pairing session each month as our way of giving back to the Laravel community? Just pick the community option.