Stochastic games are two-player repeated games in which a state variable follows a Markov chain controlled by both players. The model was initially proposed by Shapley (1953) who proved the existence of the discounted values. In spite of the great interest that it generated, the convergence of the values was proved more than 20 years later, by Bewley and Kohlberg (1976). The importance of this limit raised a few years later when Mertens and Neyman (1981) proved it to be a deep and robust notion.  A characterisation has been missing since then. In this paper, we provide one.