The adopted son of Ray Brown and Ella Fitzgerald, he was born in New York City, to Fitzgerald's half-sister Frances. Nat King Cole, Louis Armstrong, Frank Sinatra and many others were regular visitors during his childhood. After moving to California when he was 10, Ray discovered a passion for the drums and for singing. He attended Beverly Hills High School where he sang with school groups which toured at local festivals and hospitals. His father arranged for him to study with jazz percussionists Bill Douglass and Chuck Flores. Despite his jazz roots he admits that his interest in performing rock music was an act of rebellion. "I used to drive my parents crazy with this. I told my father a few years before he died, you know a lot of times I played that music 'cause I just wanted to see the look on your face.". His mother's 1964 single "Ringo Beat" was inspired by her son's interest in rock music. He moved to Seattle in 1971, studying with another drummer, Bill Coleman Sr. It was at this time that Brown began writing songs in addition to playing drums and piano. In the late 1980s Brown toured the Pacific Northwest, Canada, Alaska, Japan, Korea and Guam performing in tours for the United States Department of Defense. This is when his daughter, Alice, was conceived and born in Ketchikan, Alaska to Rebecca Judd. Like his famous parents, Brown says he loves performing on the road. He states,
"I love to tour. I've been able to see a lot of the world and I just absolutely enjoyed being able to be with different types of people, different cultures and to see the world. It's just kind of broadened who I am as a person, which eventually will filter down into the music, the influences."
In 2001, Brown recorded his debut album Slow Down for Love on SRI Jazz, which reached the top 50 of the Gavin Report. His second album, Committed from the Heart, was released in 2003. His sound has been described as an adult contemporary pop sound layered with R&B. Brown wrote and arranged all the songs on both albums. Additionally in 2003, Brown debuted on the Las Vegas Strip, performing five nights a week in the Le Bistro Theater at the Riviera hotel and casino. In 2007 he released the album Stand by Me. In September 2008, Ray Brown Jr. released a duets album, Friends and Family, produced by Shelly Liebowitz, on SRI Jazz, a division of SRI Records. Vocal artists on this album include Jane Monheit, Melba Moore, James Moody, Maria Muldaur, Dr. John, Dionne Warwick, Freda Payne, Sophie B. Hawkins, Paul Williams, Dr. Lonnie Smith, Dave Somerville, Kim Hoyer, and Sally Kellerman. Additional instrumental performances appear on some of the album tracks with artists David "Fathead" Newman, and jazz vibraphonist Terry Gibbs. Also included on the album is Brown's second daughter who went by the stage name 'Haylee' singing "A-Tisket, A-Tasket", which was made famous by her grandmother. There is also a bonus track with Brown's parents in a live version of "How High the Moon" with Ella Fitzgerald and Brown Jr. on vocals and Ray Brown playing bass. In 1998, Brown attended the Juvenile Diabetes Foundation ceremony honoring his mother, Ella Fitzgerald. In 2007, he appeared in a BBC documentary talking about his mother entitled Ella Fitzgerald: First Lady of Song. Also in 2007, he attended the ceremony for the Ella Fitzgerald stamp produced by the United States Postal Service. He traveled to events commemorating what would have been her 90th birthday. On 30 May 2020, he appeared in a BBC documentary entitled Ella Fitzgerald: Just One of Those Things.