I personally think Epstein's choice of material plus the fact that George sang a few too many lead vocals were the main contributors to the rejection.
The Star Club set list has a much higher proportion of rock n roll songs than the Decca tape - the Decca tape is weighted heavily towards the cabaret/show tunes. Even the one Chuck Berry song done for Decca - Memphis - isn't an all-out rocker in the same vein as the other Berry songs in the Beatles early repertoire, like Roll Over Beethoven or Sweet Little Sixteen.
And as much as I like George's voice he wasn't a strong lead vocalist at that point.
The fact that they had driven through a snow storm from Liverpool to London on New Years Eve can't have helped either!
Mike Smith of Decca had gone to Liverpool a few weeks before the studio audition and seen the Beatles play live - that would surely have been a more representative view of the band, and Smith was impressed enough to write that he was certain that Decca could put the Beatles to good use.
Though McCartney now insists they weren't very good at that time, they must have been an incredibly tight live band - to have recorded that audition under difficult circumstances, in a very short space of time, with all songs done in a single take, it's incredible that it's not full of mistakes. How many young bands today would put up with such treatment and manage to record something that was still worth listening to almost 50 years later?