I did some more work on this and was able to get it to work this afternoon. It turns out I forgot to do some filtering and was getting old responses that were missing data. I added a date after filter. I also changed it so the primary field the report was built on was Contacts with Survey Responses and Tickets added as secondary fields. After these changes, I did start seeing all the data I wanted to see.