That what happens with an audit. You get a letter (I got 2 of them 3 years ago) saying there was a dispute over your tax bill. It didn't mean I was avoiding paying taxes, it meant they were questioning a filing and wanted my records and it went back 5 or 6 years. They announced they were disputing (based on their calculations) what I owed. My accountant took care of it, they were wrong and what it amounted to was 'someone else' made an incorrect filing that, when corrected, exonnerated me. So someone could have correctly stated that I 'had a tax issue' (true) and from there jumped the shark to, 'I was avoiding taxes' (untrue) - which would be a flimsy grasp for something that isn't. Nonetheless, people do this.
The reason it goes back so far (and makes it look like you've been avoiding for so long, is that the IRS (like Social Security) is soo fucking slow, they're years behind. That's also why you need to keep ALL tax records fopr at LEAST 7 years.