Feature Model-Guided Online Reinforcement ...
Document type :
Communication dans un congrès avec actes
Title :
Feature Model-Guided Online Reinforcement Learning for Self-Adaptive Services
Author(s) :
Metzger, Andreas [Auteur]
PALUNO - The Ruhr Institute for Software Technology [PALUNO]
Quinton, Clément [Auteur]
Self-adaptation for distributed services and large software systems [SPIRALS]
Adam-Mann, Zoltan [Auteur]
PALUNO - The Ruhr Institute for Software Technology [PALUNO]
Baresi, Luciano [Auteur]
Politecnico di Milano [Milan] [POLIMI]
Pohl, Klaus [Auteur]
PALUNO - The Ruhr Institute for Software Technology [PALUNO]
PALUNO - The Ruhr Institute for Software Technology [PALUNO]
Quinton, Clément [Auteur]
Self-adaptation for distributed services and large software systems [SPIRALS]
Adam-Mann, Zoltan [Auteur]
PALUNO - The Ruhr Institute for Software Technology [PALUNO]
Baresi, Luciano [Auteur]
Politecnico di Milano [Milan] [POLIMI]
Pohl, Klaus [Auteur]
PALUNO - The Ruhr Institute for Software Technology [PALUNO]
Conference title :
International Conference on Service Oriented Computing
City :
Dubai
Country :
Émirats arabes unis
Start date of the conference :
2020-12-14
Journal title :
Proceedings of the 18th International Conference on Service-Oriented Computing
English keyword(s) :
Adaptation
Reinforcement learning
Feature model
Cloud service
Reinforcement learning
Feature model
Cloud service
HAL domain(s) :
Informatique [cs]/Génie logiciel [cs.SE]
English abstract : [en]
A self-adaptive service can maintain its QoS requirements in the presence of dynamic environment changes. To develop a self-adaptive service, service engineers have to create self-adaptation logic encoding when the service ...
Show more >A self-adaptive service can maintain its QoS requirements in the presence of dynamic environment changes. To develop a self-adaptive service, service engineers have to create self-adaptation logic encoding when the service should execute which adaptation actions. However, developing self-adaptation logic may be difficult due to design time uncertainty ; e.g., anticipating all potential environment changes at design time is in most cases infeasible. Online reinforcement learning addresses design time uncertainty by learning suitable adaptation actions through interactions with the environment at runtime. To learn more about its environment, reinforcement learning has to select actions that were not selected before, which is known as exploration. How exploration happens has an impact on the performance of the learning process. We focus on two problems related to how a service's adaptation actions are explored: (1) Existing solutions randomly explore adaptation actions and thus may exhibit slow learning if there are many possible adaptation actions to choose from. (2) Existing solutions are unaware of service evolution, and thus may explore new adaptation actions introduced during such evolution rather late. We propose novel exploration strategies that use feature models (from software product line engineering) to guide exploration in the presence of many adaptation actions and in the presence of service evolution. Experimental results for a self-adaptive cloud management service indicate an average speed-up of the learning process of 58.8% in the presence of many adaptation actions, and of 61.3% in the presence of service evolution. The improved learning performance in turn led to an average QoS improvement of 7.8% and 23.7% respectively.Show less >
Show more >A self-adaptive service can maintain its QoS requirements in the presence of dynamic environment changes. To develop a self-adaptive service, service engineers have to create self-adaptation logic encoding when the service should execute which adaptation actions. However, developing self-adaptation logic may be difficult due to design time uncertainty ; e.g., anticipating all potential environment changes at design time is in most cases infeasible. Online reinforcement learning addresses design time uncertainty by learning suitable adaptation actions through interactions with the environment at runtime. To learn more about its environment, reinforcement learning has to select actions that were not selected before, which is known as exploration. How exploration happens has an impact on the performance of the learning process. We focus on two problems related to how a service's adaptation actions are explored: (1) Existing solutions randomly explore adaptation actions and thus may exhibit slow learning if there are many possible adaptation actions to choose from. (2) Existing solutions are unaware of service evolution, and thus may explore new adaptation actions introduced during such evolution rather late. We propose novel exploration strategies that use feature models (from software product line engineering) to guide exploration in the presence of many adaptation actions and in the presence of service evolution. Experimental results for a self-adaptive cloud management service indicate an average speed-up of the learning process of 58.8% in the presence of many adaptation actions, and of 61.3% in the presence of service evolution. The improved learning performance in turn led to an average QoS improvement of 7.8% and 23.7% respectively.Show less >
Language :
Anglais
Peer reviewed article :
Oui
Audience :
Internationale
Popular science :
Non
ANR Project :
Collections :
Source :
Files
- https://hal.archives-ouvertes.fr/hal-02982029v2/document
- Open access
- Access the document
- https://hal.archives-ouvertes.fr/hal-02982029v2/document
- Open access
- Access the document
- https://hal.archives-ouvertes.fr/hal-02982029v2/document
- Open access
- Access the document
- document
- Open access
- Access the document
- ICSOC2020.pdf
- Open access
- Access the document