"Your Surname Tells My Shirt's Color!"

Mr. Yellow, Mr. Black and Mr. Brown, three best friends since kindergarten meet in a function after 5 years. The three of them are wearing either a Yellow, Black or Brown shirt.
After giving each other a friendly hug, Mr. Black says, "Hey, did you notice that we are wearing a different colored shirt than our names!"
The man wearing a Brown shirt said, "Wow I certainly did not notice that but you are right Mr. Black."
Based on this conversation, can you find our who was wearing which colored shirt? 

 
Know color of each person's shirt here!