let's now speak briefly about stochastic 现在让我们简要地谈论一下随机
games this is a topic that lends itself 游戏这是一个适合自己的话题
to a very long discussion and quite a 经过很长时间的讨论，
complicated one but we'll touch on the 复杂的一个，但我们将涉及
main points and position it to the 要点并定位到
landscape of topics we're discussing and 我们正在讨论的主题以及
so the strategy the striking point are 所以策略的重点是
repeated games as we know a repeated 我们知道重复的游戏
game is simply a game in normal form for 游戏只是一种普通形式的游戏
example that will repeat over and over 这个例子会一遍又一遍地重复
again for example we play prisoner's 再例如，我们扮演囚犯
dilemma once twice three times maybe 两次困境两次，也许
finite time maybe an infinite number of 有限的时间也许是无限的
time then we accumulate all the rewards 时间，然后我们积累所有的奖励
all the time to sum over all rewards 一直在总结所有奖励
stochastic gamely is a generalization of 随机博弈是对
it where we play games repeatedly but 它是我们反复玩游戏的地方，但是
not necessarily the same game so we play 不一定是同一游戏，所以我们玩
a game depending on how we played that 取决于我们如何玩的游戏
game let's say prisoner's dilemma we 游戏让我们说囚徒的困境
each got some payoff but depending on 每个人都有一些回报，但取决于
how we play the game we also probable 我们如何玩游戏，我们也可能
probabilistically transition to some 概率性地过渡到一些
other game play that in turn and the 依次进行的其他游戏和
process continues a graphical way to 过程继续以图形方式
look at it is here if if this here is a 看这是这里，如果这是一个
repeated game where you play the same 重复游戏，您玩同样的游戏
game over and over again here you play 在这里玩一遍又一遍的游戏
the game and then if you happen to play 游戏，然后如果您碰巧玩
this your transition to this game if you 这是您向该游戏的过渡，如果您
happen to just play this you maybe 碰巧只是玩这个，你也许
transition to the same game if you play 如果您玩的话，过渡到同一个游戏
this you change your transition here if 如果你改变这里的过渡
you play this maybe you'll play this 你玩这个也许你会玩这个
game again and so on from each game you 再玩一次，依此类推
transition probabilistically to toe it 概率过渡到脚趾
to other games this is a stochastic game 对其他游戏来说这是一个随机的游戏
formally speaking it's it's the 正式来说是
following tuple it's a lot of notation 以下元组有很多符号
but the concept is exactly so we have is 但是这个概念正是我们所拥有的
a finite set of states Q we have a set 有限的一组状态Q我们有一组
of players we have a set of actions 我们有一系列行动
where 哪里
actions are available to two specific 有两个特定的操作可用
players of a sub I is the action 子我是玩家的动作
available player I and then we have two 可用的球员我，然后我们有两个
two functions we have the transition 我们有两个功能过渡
probability function so depending on the 概率函数，因此取决于
state were in and on the actions we took 陈述我们采取的行动
we move to each of any of the other 我们移到其他任何一个
states or the very same state with a 状态或与一个状态完全相同的状态
certain probability as governed by this 受此约束的特定概率
probability distribution and and 概率分布和
similarly a reward is the reward 同样，奖励就是奖励
function which tells us if in a certain 告诉我们如果
State a certain action profile was taken 说明采取了某种行动
by the agents then this is a reward to 由代理商，那么这是对
to that particular agent to eat each of 给那个特工吃
the agents so our sub I is a reward to 代理商，所以我们的子我是对
two agent I that's the formal definition 我是正式的两个代理人
notice that it's sort of assumes 请注意，这是一种假设
implicitly that you have the same action 暗含你有相同的动作
spaces here but you could define it 这里有空格，但您可以定义它
otherwise it simply would involve more 否则，它将涉及更多
notations or nothing inherently 表示法或本来就不存在
important about the action spaces being 关于动作空间很重要
the same in the different games within 在不同的游戏中相同
this stochastic game so just a few final 这个随机的游戏，所以最后几场
comments on it first of all as we as we 首先像我们一样评论它
saw this obviously general generalizes 看到这显然可以概括
the notion of a repeated game but it 重复游戏的概念，但它
also generalizes the notion of an MDP or 也概括了MDP或
a Markov decision process if a if a 马尔可夫决策过程，如果
sarcastic game if a repeated game is a 如果重复的游戏是一个讽刺游戏
stochastic game with only one game a 一个只有一个游戏的随机游戏
Markov decision process on MVP is a game MVP的马尔可夫决策过程是一个游戏
with only one player and so you have 只有一个玩家，所以你有
States there where the agents take agent 代理商所在的国家
takes an action receives a remediate 采取行动得到补救
reward and probably move to some other 奖励并可能转移到其他人
state and the only difference is that he 状态，唯一的区别是他
is the only actor in the setting I 是背景中我唯一的演员
mentioned this because well empty peas 提到这个是因为豌豆很好
have been studied 已经研究过
naturally in a variety of disciplines 自然地在各种学科中
from optimization to computer science to 从优化到计算机科学，再到
pure math and beyond but also these two 纯粹的数学，以及超越这两个
perspectives of generalization repeated 泛化观点重复
games and of MVPs give you a sense for 游戏和MVP可以带给您
the theory and investigations into 理论与研究
stochastic games so from repeated games 重复游戏中的随机游戏
we inherit the definitions of different 我们继承了不同的定义
ways of the aggregating rewards over 汇总奖励的方式
time you can have limited average 有限的平均时间
rewards future discounted rewards 奖励未来的折扣奖励
whereas from the literature on 而根据文献
optimization and on MVPs we get notions 优化，在MVP上我们有了概念
such as stationarity and Markovian 例如平稳性和马尔可夫式
strategies these have to do with we also 这些与我们有关的策略
have notions of reach ability about the 有关于
structure of the underlying transition 基本过渡的结构
probability and so again these are 概率，这些又是
issues that are involved that we won't 我们不会涉及的问题
get to into more in this lecture but at 在本讲座中深入了解，但在
least we flagged their existence 至少我们标记了它们的存在