In the English language, at least American Slang, yes 'being yella or yellow', does mean cowardly, used in context. You can see this in old American westerns, but by today's standards 'yellow' isn't really used in this manner, and I think there is an inference of racisms towards the Asian population if use in a particular way. So while you'll see reference to White or Black people, no one really uses Yellow people, it would be considered rude at best, and it relates back to the yella being related to cowardly.
Not to derail the thread, and I think this is on topic, the term 'queer' has / had two meanings - 1) odd, or against the grain 2) meaning gay.
1) This is the usage in poker, where you have two pair, but the board pairs and 'queers' or disqualifies your hand in some way - example:
V: Q3 H: AA
B: Q 3 4 10 4
The Villain flopped 2 pair but got queered.
The second usage use to be derogatory and can still be used that way via inflection or implied, but is used just as much or more used today to define a specific type of gay, as I understand it.
Disclaimer, none of this is meant to be political or rude or offend anyone, just discussion and educational purposes.