Born and raised in Wenatchee, Washington, Worley grew up in a family with ten children. He attended Wenatchee High School, where he was a three-sport standout in football, basketball, and baseball, finishing as the school's fourth all-time leading scorer in basketball. Worley stated he was lightly recruited and "I was surprised that Washington even offered me a scholarship. I was not what you'd call a widely recruited athlete. I was an all-nothing in high school." Worley was selected as the "Panther Best" award recipient his senior year and is a member of the Wenatchee High School Athletic Hall of Fame.
College career
After playing split end during his freshman year at Washington in 1964, coach Ed Peasley told head coachJim Owens that "Al Worley will never play varsity football at the University of Washington." Dave Williams, Washington's All-America tight end and a future No. 1 NFL draft pick, disagreed, stating "That kid in the red shirt gives me more trouble than anybody else." Worley was a redshirt in 1965, suffering a broken hand. He saw playing time in the secondary in the 1966 and 1967 seasons, progressing to a full-time starter for the 1968 season as a fifth-year senior. For the 1968 team, Worley recorded three interceptions in Washington's 21-17 victory at Wisconsin, two with Wisconsin inside Washington's 10-yard line, setting a conference single-game record. Against Idaho at Husky Stadium, Worley recorded four of Washington's school-record eight interceptions in a 37-7 victory, which re-established his conference single-game record. Worley finished the 1968 season with an NCAA record 14 interceptions. In 2014, his record was tied by Gerod Holliman of Louisville. Worley's record season was accomplished in a 10-game season, while Holliman played in a 12-game regular season plus a bowl game and with his team facing twice as many pass attempts.
Worley was a part-time Washington assistant and a substitute teacher in Seattle. Worley then became a full time coach at Northern Arizona from 1971 to 1974 under head coach Ed Peasley. He was later an assistant at Portland State under head coach Mouse Davis. In 1979, Worley became head coach of the Yokosuka Base Seahawks, a U.S. Navy Service team in Japan. Worley settled in Hawaii and worked as a facilities and projects manager. In 2015 and 2016, Worley was nominated for induction to the College Football Hall of Fame.