Not only George's songs had Indian music prevalent. There was indian music also in "Baby You're A Rich Man."
Now. Its my assumption that George Harrison never really learned how to play, other than a few notes (he played sitar on "Noreigian Wood"). Just like they had an orchestra for whenever horns or harps or violins were needed in a song, George Harrison had an indian orchestra for "Love You To," "Within You Without You," and the rest.
Nobody here is going to tell me something different, are they?