Description

Book Synopsis
This book is an up-to-date, unified and rigorous treatment of theoretical, computational and applied research on Markov decision process models. The concentration of the book is on infinite-horizon discrete-time models, and it also discusses arbitrary state spaces, finite-horizon and continuous-time discrete-state models.

Table of Contents
Preface.

1. Introduction.

2. Model Formulation.

3. Examples.

4. Finite-Horizon Markov Decision Processes.

5. Infinite-Horizon Models: Foundations.

6. Discounted Markov Decision Problems.

7. The Expected Total-Reward. Criterion.

8. Average Reward and Related Criteria.

9. The Average Reward Criterion-Multichain and Communicating Models.

10. Sensitive Discount Optimality.

11. Continuous-Time Models.

Afterword.

Notation.

Appendix A. Markov Chains.

Appendix B. Semicontinuous Functions.

Appendix C. Normed Linear Spaces.

Appendix D. Linear Programming.

Bibliography.

Index.

Markov Decision Processes

    Product form

    A Paperback / softback by Martin L. Puterman

    £116.96

    Includes FREE delivery

    RRP £129.95 – you save £12.99 (9%)

    Order before 4pm tomorrow for delivery by Wed 21 Oct 2026.

    1 in stock

      Trusted by thousands of customers. See 2,385+ Customer Reviews

      View other formats and editions of Markov Decision Processes by Martin L. Puterman

      Book details

      Publisher John Wiley & Sons Inc
      Published 25 February 2005
      ISBN-13 9780471727828
      978-0471727828
      ISBN-10 0471727822
      Also in:
      Mathematics

      Description

      Book Synopsis
      This book is an up-to-date, unified and rigorous treatment of theoretical, computational and applied research on Markov decision process models. The concentration of the book is on infinite-horizon discrete-time models, and it also discusses arbitrary state spaces, finite-horizon and continuous-time discrete-state models.

      Table of Contents
      Preface.

      1. Introduction.

      2. Model Formulation.

      3. Examples.

      4. Finite-Horizon Markov Decision Processes.

      5. Infinite-Horizon Models: Foundations.

      6. Discounted Markov Decision Problems.

      7. The Expected Total-Reward. Criterion.

      8. Average Reward and Related Criteria.

      9. The Average Reward Criterion-Multichain and Communicating Models.

      10. Sensitive Discount Optimality.

      11. Continuous-Time Models.

      Afterword.

      Notation.

      Appendix A. Markov Chains.

      Appendix B. Semicontinuous Functions.

      Appendix C. Normed Linear Spaces.

      Appendix D. Linear Programming.

      Bibliography.

      Index.

      Recently viewed products

      © 2026 Book Curl

        • American Express
        • Apple Pay
        • Diners Club
        • Discover
        • Google Pay
        • Maestro
        • Mastercard
        • PayPal
        • Shop Pay
        • Union Pay
        • Visa

        Login

        Forgot your password?

        Don't have an account yet?
        Create account