Learning another language is difficult. I often hear people classify Mandarin, Cantonese, Thai, and others as "the hardest language" or "one of the hardest languages." But if you were a native speaker of, say, Lao, and attempted to learn Thai, it would be vastly easier than if you were a native English speaker. In my mind, it's not really about how inherently difficult a certain language is, but rather about how different the language you would like to learn is from the language(s) you already know. Relative similarity or difference in the phonology, morphology, and syntax of a language is what counts. Are they typologically similar? That is, do they share many features--word order, for example?
Thai and English have the same basic word order: what we call SVO (subject+verb+object). That means that where in English we say "I eat rice," Thais say ฉันกินข้าว. The subject, followed by the verb, followed by the object (when there is one). This is a huge leg up for English speakers! Sure, you have to remember to say อาหารอร่อย [food+delicious] instead of "delicious food", but if you're an American, where it seems like everyone is exposed to Spanish, you may already know that it shares this word order of noun+modifier with Thai, so it doesn't seem so strange.
Then compare Thai word order with Japanese, which has SOV word order: "I rice eat", or Tagalog, which has VSO word order: "eat I rice". See how you've already got a head start?
Apart from that, however, English and Thai diverge pretty quickly. They are both generally classified as analytic languages, meaning that words tend to stand alone and that word order is usually significant for conveying a given meaning, but English has synthetic features, like attaching /s/ for plurals, or other various prefixes and suffixes.
The phonology is also quite different in many ways, which I plan to tackle in the (long overdue) continuation of my Improve Your Accent series. And I probably needn't even mention tones. If you want to boggle the minds of unsuspecting English speakers, just bust out with a minimal quintet of the five tones: มา หม่า ม่า ม้า หมา (or the classic tongue twister about whether the new wood is burning: ไม้ใหม่ไหม้ไหม ไม่ไม้ใหม่ไม่ไหม้). The flip side of this is that English has crazy spelling, some verb conjugation, many sounds that are foreign to the Thai speaker, including many difficult consonant clusters, etc.
Mastering Thai requires a commitment to learn to speak in new ways. Moving your tongue in new ways, making sounds in different ways, internalizing new word orders, and stepping outside of the way you're used to thinking about the world. It's a difficult task that requires significant self-awareness and dedication. It's not for the faint of heart, but the rewards of knowledge for those who persevere are immense.