BZPEERReading, trust and collaboration for research
Menu

Discover

GraphFollow connections around a paper, topic or saved view.

Workspaces

HomePick up where your research flow left off.InboxHandle requests, replies and notifications from one place.CollectionsCurate reading lists, evidence sets and shareable dossiers.ResearchSee your private provenance graph, trail and imports.CollaborationMove from discovery into focused discussion rooms.PublishDraft, preview and publish full-text research articles.

Network

ExpertsFind specialists worth following or asking for help.CommunitiesJoin research circles organized around shared work.PartnersSee external organizations active around the same topics.

Opportunities

ListingsBrowse roles, calls and collaborations with clearer fit signals.

Account

Log inReturn to your reading and collaboration workspace.Create accountStart saving work, following people and joining teams.
Log inCreate account

Researcher profile

Lukas Beckenbach

Lukas Beckenbach contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate
eess.SYmath.OCSystems and Controlmath.DSMachine Learning

Trust snapshot

Quick read

Trust 21 - EmergingVerification L1Unclaimed author
7works
0followers
5topics
4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Identity and collaboration

How to connect with this researcher

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Log in to claim

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Research graph

See the researcher in context

Open full explorer

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

7 published item(s)

preprint2022arXiv

A stabilizing reinforcement learning approach for sampled systems with partially unknown models

Reinforcement learning is commonly associated with training of reward-maximizing (or cost-minimizing) agents, in other words, controllers. It can be applied in model-free or model-based fashion, using a priori or online collected system data to train involved parametric architectures. In general, online reinforcement learning does not guarantee closed loop stability unless special measures are taken, for instance, through learning constraints or tailored training rules. Particularly promising are hybrids of reinforcement learning with "classical" control approaches. In this work, we suggest a method to guarantee practical stability of the system-controller closed loop in a purely online learning setting, i.e., without offline training. Moreover, we assume only partial knowledge of the system model. To achieve the claimed results, we employ techniques of classical adaptive control. The implementation of the overall control scheme is provided explicitly in a digital, sampled setting. That is, the controller receives the state of the system and computes the control action at discrete, specifically, equidistant moments in time. The method is tested in adaptive traction control and cruise control where it proved to significantly reduce the cost.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Approximate infinite-horizon predictive control

Predictive control is frequently used for control problems involving constraints. Being an optimization based technique utilizing a user specified so-called stage cost, performance properties, i.e., bounds on the infinite horizon accumulated stage cost, aside closed-loop stability are of interest. To achieve good performance and to influence the region of attraction associated with the prediction horizon, the terminal cost of the predictive controller's optimization objective is a key design factor. Approximate dynamic programming refers to one particular approximation paradigm that pursues iterative cost adaptation over a state domain. Troubled by approximation errors, the associated approximate optimal controller is, in general, not necessarily stabilizing nor is its performance quantifiable on the entire approximation domain. Using a parametric terminal cost trained via approximate dynamic programming, a stabilizing predictive controller is proposed whose performance can directly be related to cost approximation errors. The controller further ensures closed-loop asymptotic stability beyond the training domain of the approximate optimal controller associated to the terminal cost.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Performance bounds of adaptive MPC with bounded parameter uncertainties

Model predictive control is a control approach that minimizes a stage cost over a predicted system trajectory based on a model of the system and is capable of handling state and input constraints. For uncertain models, robust or adaptive methods can be used. Because the system model is used to calculate the control law, the closed-loop behavior of the system and thus its performance, measured by the sum of the stage costs, are related to the model used. If it is adapted online, a performance bound is difficult to obtain and thus the impact of model adaptation is mostly unknown. This work provides a (worst-case) performance bound for a linear adaptive predictive control scheme with a specific model parameter estimation. The proposed bound is expressed in terms of quantities such as the initial system parameter error and the constraint set, among others and can be calculated a priori. The results are discussed in a numerical example.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

On performance bound estimation in NMPC with time-varying terminal cost

Model predictive control (MPC) schemes are commonly designed with fixed, i.e., time-invariant, horizon length and cost functions. If no stabilizing terminal ingredients are used, stability can be guaranteed via a sufficiently long horizon. A suboptimality index can be derived that gives bounds on the performance of the MPC law over an infinite-horizon (IH). While for time-invariant schemes such index can be computed offline, less attention has been paid to time-varying strategies with adapting cost function which can be found, e.g., in learning-based optimal control. This work addresses the performance bounds of nonlinear MPC with stabilizing horizon and time-varying terminal cost. A scheme is proposed that uses the decay of the optimal finite-horizon cost and convolutes a history stack to predict the bounds on the IH performance. Based on online information on the decay rate, the performance bound estimate is improved while the terminal cost is adapted using methods from adaptive dynamic programming. The adaptation of the terminal cost leads to performance improvement over a time-invariant scheme with the same horizon length. The approach is demonstrated in a case study.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

A reinforcement learning method with closed-loop stability guarantee

Reinforcement learning (RL) in the context of control systems offers wide possibilities of controller adaptation. Given an infinite-horizon cost function, the so-called critic of RL approximates it with a neural net and sends this information to the controller (called "actor"). However, the issue of closed-loop stability under an RL-method is still not fully addressed. Since the critic delivers merely an approximation to the value function of the corresponding infinite-horizon problem, no guarantee can be given in general as to whether the actor's actions stabilize the system. Different approaches to this issue exist. The current work offers a particular one, which, starting with a (not necessarily smooth) control Lyapunov function (CLF), derives an online RL-scheme in such a way that practical semi-global stability property of the closed-loop can be established. The approach logically continues the work of the authors on parameterized controllers and Lyapunov-like constraints for RL, whereas the CLF now appears merely in one of the constraints of the control scheme. The analysis of the closed-loop behavior is done in a sample-and-hold (SH) manner thus offering a certain insight into the digital realization. The case study with a non-holonomic integrator shows the capabilities of the derived method to optimize the given cost function compared to a nominal stabilizing controller.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Model Predictive Control of a Food Production Unit: A Case Study for Lettuce Production

Plant factories with artificial light are widely researched for food production in a controlled environment. For such control tasks, models of the energy and resource exchange in the production unit as well as those of the plant's growth process may be used. To achieve minimal operation cost, optimal control strategies can be applied to the system, taking into account the availability of resources by control reference specification. A particular advantage of model predictive control (MPC) is the incorporation of constraints that comply with actuator limitations and general plant growth conditions. In this work, a model of a production unit is derived including a description of the relation between the actuators' electrical signals and the input values to the model. Furthermore, a preliminary model based state tracking control is evaluated for production unit containing Lettuce. It could be observed that the controller is capable to track the reference while satisfying the constraint under changing weather conditions and resource availability.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Model predictive control with stage cost shaping inspired by reinforcement learning

This work presents a suboptimality study of a particular model predictive control with a stage cost shaping based on the ideas of reinforcement learning. The focus of the suboptimality study is to derive quantities relating the infinite-horizon cost function under the said variant of model predictive control to the respective infinite-horizon value function. The basis control scheme involves usual stabilizing constraints comprising of a terminal set and a terminal cost in the form of a local Lyapunov function. The stage cost is adapted using the principles of Q-learning, a particular approach to reinforcement learning. The work is concluded by case studies with two systems for wide ranges of initial conditions.

0 citations0 reviewsNo code linkedOpen access