Skip to the content.

I’m an adjunct professor at Department of Computing Science, University of Alberta. My research interest is model-based reinforcement learning with a recent focus on stochastic gradient methods, step-size adaptation, and sparse representation learning.

I studied reinforcement learning at the RLAI lab from 2008 to 2014 in a Ph.D program at the same department. I worked on offline model-based approximate policy iteration, multi-step linear Dyna-style planning, a novel reinforcement learning perspective for PageRank, and Webpage ranking and Web search. I interned with Yahoo! Sunnyvale in 2013 and won a championship (CEO award) for “TrendingNow” project that detects trending topics in Yahoo search engine. I joined NCSoft game studio in San Francisco in 2016 working on reinforcement learning for mobile games. I joined Huawei in 2017 and founded joint lab projects between University of Alberta and Huawei.

Education:

Thesis

Publication

Here is the testing performance of a low-quantile $Q_{0.1}$ driving in CARLA (A 3D driving simulator) in a safety-based reward function as we have more and more samples. This motivates “Distributional Reinforcement Learning”!

hi
hi

Reviewing

AAAI, ICML, NIPS, UAI, ADPRL, AIStat, CIKM, WWW.

Robocup Soccer

I was a member of TsinghuAeolus (soccer simulation team at Tsinghua University) for World Cup Simulation League. Here shows a game of our team TsinghuAeolus playing against Everest in the final match at Robocup 2003. Our TsinghuAeolus was based on hierarchical reinforcement learning. Decisions the player has to make include: dribbling, passing, running, shooting, goal keeping, positioning (team position), resting, defending, etc. A high-level policy learns how to select a decision, and a low-level controller executes the selected decision. Both the high-level and low-level controller were learned using reinforcement learning.

Tetris

I developed a policy iteration algorithm to play a Tetris game. In this game, there are only hard shapes: “S” and “Z”. The player was trained with data of randomly playing the game.

Contact

hengshu1 “At@” ualberta.ca

Life

I live in Edmonton with my family, Dongcui, Alex, Marissa, and Dylan. Our favourite activity is to follow our cat Benji for a spiritual trail exploration.