He is really a strong skilled swordsman, even though he looks beautiful (cute?)cool (or attractive or stylish or amazing).

See a lot of variations of interpretations. Closest to Japanese in my opinion would be beautiful = 美しい (women) カッコイイ = cool / stylish / attractive (men). Men can be beautiful, but generally if you called a man in Japan 美しい it would be kind of the same as calling a man beautiful in US English. There is so much nuance and it depends on the situation who is talking to who, it's really hard to pin down without specific context. Hope this helps a little bit.

He is a really cold character, but after he meets Tanjiro, the main character, his cold personality gradually changes to mild.. (or: he gradually begins to warm up to Tanjiro.)

He is really a strong skilled swordsman, even though he looks beautiful (cute?).

Your sentence is fine, however here are some other ways to say it more naturally in English:

[...] even though he has a pretty face.
[...] even though he is a pretty boy.

For male characters, the phrases "pretty boy" and "pretty face" are more often used than "beautiful", however both have the same meaning, so it's okay to use any of them.

He doesn't show any emotion, and the way he talks is blunt and curt.

if the part after the comma is a complete thought, then it is preceded by a coordinating conjunction like and, but, so, for, yet, et cetera.

if it is not a complete thought, the comma is unnecessary

He is a really cold character, but after he meets Tanjiro, the main character, his cold personality gradually changes tobecomes mild.

or "gradually changes to a mild one"

"changes to mild" sounds a bit uncommon to me

He is really a strong skilled swordsman, even though he looks beautiful (cute?)pretty.

I might say something like that he looks delicate or gentle

I wasn't really into Demon Slayer during Season 1, but when I watched Season 4, I was completely hooked.

He is a really cold character, but after he meets Tanjiro, the main character, his cold personality gradually changes to mild.

He is really a strong and skilled swordsman, even though he looks beautiful (cute?)girlish

When describing people:

- Beautiful is normally used for women that look like conventionally attractive women (Men would normally be called "handsome")
- Cute is normally used for children and babies

I don't think either quite fit but I googled the character and think "girlish" is probably the best adjective here. It's normally used for people that look like girls that aren't expected to be (e.g. women that look much younger than they are, men and boys that look childish and effeminate). However, unlike e.g, 美少年, it's a little insulting to the person described that way. This is kind of a cultural thing, apart from certain groups (e.g. LGBT people) that kind of appearance isn't usually considered to be something people want in English speaking countries, so basically all the adjectives to describe it have a negative connotation. They're not slurs or anything, just best kept to describing fictional characters or celebrities rather than, e.g., your coworker.

He is a really a strong skilled swordsman, even though he looks beautiful (cute?)very pretty

I suggested "pretty" because it's a bit more in between beautiful and pretty. The first half of your sentence isn't wrong per se but it is just unnatural so I changed "a" and "really" around to make a more natural sentence.