As it turns out there is no audit trail to see which specific criteria points set a score in a Company/Contact record.
You can only review the rules of the score itself to know how it would be reached and then review those factors. This means there’s a lot of manual checking and I feel that a log of all the criteria’s per record would be beneficial here.