### The Optimal Reward Operator in Special Classes of Dynamic Programming Problems

David A. Freedman
Source: Ann. Probab. Volume 2, Number 5 (1974), 942-949.

#### Abstract

Consider a dynamic programming problem with separable metric state space $S$, constraint set $A$, and reward function $r(x, P, y)$ for $(x, P)\in A$ and $y\in S$. Let $Tf$ be the optimal reward in one move, for the reward function $r(x, P, y) + f(y)$. Three results are proved. First, suppose $S$ is compact, $A$ closed, and $r$ upper semi-continuous; then $T^n0$ is upper semi-continuous, and there is an optimal Borel strategy for the $n$-move game. Second, suppose $S$ is compact, $A$ is an $F_\sigma$, and $\{r > a\}$ is an $F_\sigma$ for all $a$; then $\{T^n0 > a\}$ is an $F_\sigma$ for all $a$, and there is an $\varepsilon$-optimal Borel strategy for the $n$-move game. Third, suppose $A$ is open and $r$ is lower semi-continuous; then $T^n0$ is lower semi-continuous, and there is an $\varepsilon$-optimal Borel measurable strategy for the $n$-move game.

First Page:
Primary Subjects: 49C99
Secondary Subjects: 60K99, 90C99, 28A05
Full-text: Open access

Permanent link to this document: http://projecteuclid.org/euclid.aop/1176996559
Digital Object Identifier: doi:10.1214/aop/1176996559
Mathematical Reviews number (MathSciNet): MR359819
Zentralblatt MATH identifier: 0318.49022

2013 © Institute of Mathematical Statistics

Turn MathJax Off
What is MathJax?
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 28, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 20, "end_threshold": 5, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.8623864650726318, "perplexity_flag": "head"}