I think this is entirely your call. Answers will only be automatically deleted if there are too many spam or offensive flags on them. Otherwise moderators can delete an answer if they think it is inappropriate or does not answer the question at all. In any event, all deleted answers can be seen by users with access to moderator tools (users with 10,000 rep on regular sites or 2,000 on beta sites - like this one). Your answer is at 0/-1, which is not really anything to stress over. You basically have three options:
You can delete the answer. If you delete it, as Stéphane says, reputation changes will be reverted. Upvotes are worth +10 and downvotes are worth -2. At the moment, you would gain 2 rep by deleting it, and prevent more rep loss if there would be more downvotes. On the flip side, it prevents potential rep gain from upvotes.
You can leave it as is. It could gain some upvotes, or more downvotes. By default, lowest scoring answers will be sorted at the bottom of all the other answers. Also, if it gets to a score of -3, it will be greyed out. On a side note, deleting an answer with a score of -3 gets you the Peer Pressure badge.
Your third option is to edit your answer. If you think you can improve it based on comments or new knowledge, you can always edit your own posts. If your answer is improved enough, it will likely get upvotes, and the downvote can also be taken away by its caster after an edit.
The decision is entirely yours. If your answer really needed to be deleted, it probably would have been by now. It's just a question of what you want to do with your posts.