diff --git a/docs/10.md b/docs/10.md index 868935da9135fb0c099f64c80499f5eecc9d1a35..eb6162a425dcfcf35b102772dff24d8e6f879412 100644 --- a/docs/10.md +++ b/docs/10.md @@ -21,11 +21,7 @@ $$ 我们将目标函数记为 $J(\theta)$,可以用蒙特卡洛方法估计 $J(\theta)$。我们用 $r(\tau)$ 来代表轨迹 $\tau$ 的衰减奖励总和。 $$ -J(\theta) = \mathbb{E}_{\tau\sim \pi _{\theta}(\tau)}[\sum_t \gamma^t r (s_t,a_t)] = \int \pi _ {\theta} (\tau) r(\tau) \text{d} \tau \approx \frac{1}{N}\sum_{i=1}^N \sum_{t=1}^T \gamma^t r(s_{i,t},a_{i,t}) -$$ - -$$ -\int \pi_\theta (\tau) r(\tau) \text{d} \tau +J(\theta) = \mathbb{E}_{\tau\sim \pi _{\theta}(\tau)}[\sum_t \gamma^t r (s_t,a_t)] = \int \pi _{\theta} (\tau) r(\tau) \text{d} \tau $$ $$