Integrated Process Planning and Scheduling in Commercial Smart Kitchens

10/08/2019 ∙ by Piotr Dziurzanski, et al. ∙ 0

This paper describes the possibility of applying a generic, cloud-based Optimisation as a Service facility to food cooking planning and scheduling in a commercial kitchen. We propose a chromosome encoding and customisation of the classic MOEA/D multi-objective genetic algorithm. The applicability of the proposed approach is evaluated experimentally for two scenarios different with respect to the number of cooking appliances and the amount of the ordered food. The proposed system managed to determine the trade-offs between cooking time, energy dissipation and food quality.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Fig. 1: Overall architecture of the proposed system

The term fourth industrial revolution, often abbreviated to Industry 4.0, is used to refer to the current trend of automation in manufacturing technologies. One of the promises of Industry 4.0 is related to facilitating of production of small batches of highly customised commodities [1]

. Even if not 100% correct from the orthodox standpoint, preparing dishes in a restaurant may be viewed to be consistent with the primary assumptions of Industry 4.0. The orders are released at random moments, the batches are extremely short (up to a number of people in a group) and highly customised (e.g., blue, rare, medium rare, medium or well-done steaks). Consequently, a chef is often reported as one of the most stressful jobs

[2]. Digital gastronomy aims to mitigate the chef life by enhancing traditional cooking with new capabilities rather than replacing the chef with an autonomous machine [3]. Following this trend, modern technologies are increasingly more popular in restaurants. At the moment, the Internet of Things (IoT) is used to monitor the equipment that cooks, cleans or stores food [4]

. However, in the near future technology will be probably applied to more sophisticated tasks. For example, the recent progress in developing so-called electronic tongues and noses can facilitate the automation of the process of food samples’ quality estimation

[5]. Such sensors can be used to fill the gap between recipes and actual cooking activities, identified in [6]. Even Kinect-style cameras can be applied for fine-grained recognition of kitchen activities [7]. Not only the sensors but also the actuators in smart kitchen appliances can act as things connected to the Internet, as for example a recently presented smart oven from Electrolux in which the time duration or temperature can be remotely altered [8]. Even if a certain task in a cooking process has to be done manually (e.g., adding ingredients to a pot), it can be guided by a robot speech following some recipes [9]

or even a cognitive conversational agent connected to a smart fridge

[10]. However, those systems process one recipe at a time. This is in contrast to [11], where the cooking process has been treated as an optimisation problem. That system applied the list scheduling algorithm to minimise the food time preparation and maximise food quality, benefiting from the fact that some actions can be executed in parallel, reducing the cooking duration.

The above observations encourage us to apply a generic factory reconfiguration system, described in [12], to food preparation in restaurants. In particular, similarly to a smart factory, a kitchen can receive a new order at any time. For such order, a process planning and scheduling need to be performed with no delay [13]. Process planning and scheduling are required to be re-executed in case any unexpected event occurs in the kitchen, for example, a failure of a device (treated as a smart thing) is detected [14]. If a number of devices in a kitchen is considerable, process planning and scheduling will have similar computation cost as in a smart factory, which is rather significant [15]. Between subsequent executions of the process planning and scheduling, the computational power is not needed. Consequently, the workload related to the process planning and scheduling in a restaurant follows the on-and-off workload pattern. When using the process planning and scheduling approach that can be processed in a distributed way, for example the island model of a Genetic Algorithm (GA) similarly to [16], the workload satisfies all criteria for suitability for public cloud provided in [17], namely an unpredictable load, different computational power requirements at different time intervals and horizontal scalability.

The rest of this paper is organised as follows. Section II outlines our generic service for optimisation of smart factories. Application of this service to a smart kitchen is described in Section III. In Section IV, experimental evaluation of the proposed approaches is carried out. Section V concludes the paper.

Ii Generic service for smart factories re-configuration

The class of optimisation problems analysed in this paper concerns integrated process planning and scheduling in smart kitchens performed on demand. The optimisation is carried out by the module named Optimisation Engine (OE), which is a part of the larger system presented in Figure 1. The operation of this system is triggered by the data ingested from assorted devices (things) such as smart hobs. This data is collected by the Situation Determination (SD) module, which derives the current situation of resources, products and processes, using a custom use-case situation model based on a common situation model. SD monitors raw data provided by things and the outcomes of the Predictive Analytics (PA) module and then determines the kitchen current state. In case any relevant change of the kitchen state is detected, the process plan and schedule is recomputed.

Fig. 2: Layers in the proposed approach

In the proposed system, the popular streaming platform named Kafka has been applied to the communication between modules. The messages sent via Kafka follow a well-defined textual protocol named Metrics API, which defines three types of numeric or nominal values: key objectives to be optimised, control metrics that can be mutated to obtain various candidate solutions and observable metrics informing about situations relevant to the optimisation process, such as unavailability of a certain resource. This system architecture is compliant with the scheme proposed in [18], where three-layers of an intelligent factory reconfiguration system have been identified. The lowest layer, named Specification layer, includes the knowledge description regarding the factory based on General system ontology. The middle layer, Analysing & modelling layer, includes both SD and PA modules. OE operates in the topmost layer which is named Decision layer. These layers are visualised in Figure 2. From this layered architecture, it follows that the appropriate operation of both the SD and PA modules are crucial for performing effective optimisation. However, the detailed description of these two modules is out of the scope of this paper.

The overall specification of reconfiguration capability is most readily explained via division into two component parts: the Metrics API and Optimisation Engine. The Metrics API provides a complete configuration description for the variables associated with the food order and kitchen temporal state. The elements of the configuration data schema are termed metrics, i.e. either measurable physical values corresponding to appliance sensors or else key objective (quality) measures derived from these. The chief functionality of OE is to generate a food cooking plan and schedule in response to reconfiguration requests issued by SD. The quality of the candidate plans and schedules is determined by the objective function. This function is generated automatically based on a kitchen configuration and applies a digital twin of the corresponding smart kitchen specified with Interval Algebra [13].

The optimisation used by OE is based on [12]

. There are given a set of recipes, a set of resources and an order. OE assigns resources and priorities to a multisubset (i.e. a combination with repetitions) of recipes so that the total processing time (makespan) and energy are minimised and the food quality is the highest. In genetic algorithms, candidate solutions are treated as individuals. During the optimisation process, these individuals are evolved using a set of bio-inspired operators such as selection, crossing-over and mutation. The solution to the problem considered in this paper can be then described with a chromosome whose odd genes identify the recipe to be applied and the even genes denote the priority of that recipe instance. The aim of introducing priorities is to determine the processing orders of recipes allocated to the same resource and thus to determine the temporal scheduling. This ordering does not change the amount of cooked food but can influence the makespan.

The problem analysed in this paper is characterised with multi-objective criteria. End-users should be then informed about a wide set of Pareto-optimal solutions to select the final solution based on their knowledge of the problem. The set of the alternative solutions presented to the end-users should be then diverse and, favourably, distributed over the entire Pareto front. This expectation is in line with the properties of the MOEA/D algorithm proposed by Zhang and Li in [19], which is then used in OE.

Iii Applying generic re-configuration service for smart kitchen

The optimisation process is executed after sending a serialised configuration to a predefined Kafka topic. The configuration includes the list of dishes to be cooked together with the list of the recipes and the parameters of the available hobs. An example list of recipes used in this paper is provided in Table I. For example, the list of compatible cooking zones (e.g. of the area sufficient for a certain pot or food amount) is provided in the domain of a controlled metric type, which is shown in Figure 3 for two examples related to boiled water. The resource name is composed of two parts: the actual cooking zone name and the pot type. Each recipe can be executed a number of times, thus the recipe name is followed by an instance index (e.g. suffix 0 in Boiled water A 0). As it is shown in this figure, the first recipe, Boiled water A 0 can be executed using Pot(1) and cooking zones 1-4 on hob named Hob, or not selected to be executed (i.e., No allocation). In the assumed hob, a few cooking zones can be used simultaneously to cook a dish in a larger pot, as shown in Figure 4. In this figure, four cooking zones can be used with pot(1) (i.e., the circles on the hob) independently (left column), or two upper or two lower cooking zones can be combined and used with pot(2) (middle column). Notice, that using two middle cooking zones in such a way is impossible. Finally, three upper zones can be used simultaneously with pot(3) (right column). In the configuration, all cooking zones Hob(1)-Hob(7) are provided as separate resources, but the fact that certain resources cannot be used simultaneously (e.g. Hob(1) and Hob(5)) is defined using mutual exclusiveness of resources as explained in [13].

ControlledMetricType[name=Boiled water A 0,
Boiled water A 0 allocation type ,values={Hob(1)
Pot(1), Hob(2) Pot(1), Hob(3) Pot(1), Hob(4) Pot(1),
No allocation},typ=NOMINAL]

ControlledMetricType[name= Boiled water B 0
Boiled water B 0 allocation type ,values={Hob(5)
Pot(2), Hob(6) Pot(2), No Allocation},typ=NOMINAL],
Fig. 3: Two example controlled metrics
Fig. 4: Example of possible configuration of four cooking zones to heat pots of different sizes

The observable metrics are used to denote the availability of certain resources. Due to the compound naming structure of these resources, both a certain cooking zone or a certain pot type can be signalled as being unavailable. For example, temporal lack of pot(1) should result in the unavailability of all resources that are used in combination with that pot. The first example in Figure 5 signals unavailability of a certain cooking zone. The second and third observables inform OE about the cooking time of a certain recipe (here: Boiled water A) with a certain cooking zone (here: Hob(1)) and pot (here: Pot(1)), as determined by the predictive analytics or situation determination modules. This cooking time is taken into consideration when a schedule is determined. Finally, the recipe quality can be updated based on some user feedback.

ObservableMetricType[name= Hob(6) availability,

ObservableMetricType[name=Boiled water A Hob(1)
Pot(1) start, valueType=ValueType.Integer[min=0,

ObservableMetricType[name=Boiled water A Hob(1)
Pot(1) end, valueType=ValueType.Integer[min=40,
Fig. 5: Two example observable metrics in the Electrolux use case
Optimisation took: 19.942 seconds
Schedule: Status: Succeeded.

Hob(2) Pot(1) -> [
Rice A 1_1 [35,37),
Rice A 1 [103,128),
Beef A 2_1 [188,200),
Beef A 2 [692,812),
DependentSetUp from Beef A
    to Boiled Water A [812,822),
Boiled Water A 0 [1094,1109),
DependentSetUp from Boiled Water A
    to Pasta A [1109, 1119),
Pasta A 0_1 [1297,1299),
Pasta A 0 [1299,1319)
makespan: 28:57.00
Fig. 6: Extract from an example OE report for the Electrolux use case

Iv Experimental results

In the considered scenario, the following amounts of food are required to be cooked: Boiled water - 5000g, Pasta - 1000g, Rice - 1500g, Meat (beef) - 1000g, Vegetable (potatoes) - 1000g, Mushrooms (with oil) - 500g. These values are provided to OE via Kafka and then the optimisation process is executed. An example report of the optimisation process is presented in Figure 6. As shown in the figure, the tasks are executed concurrently on all available resources, and the finish time of the last task indicates the makespan of this optimisation procedure. Note the start and end time of each task is relative to the starting time point 0. For the production that requires certain pre-cooking, subtasks are introduced and are executed before cooking the required production (i.e., task Rice A 1_1 for cooking rice using task Rice A 1). In addition, dependent setup is required when different production is executed on the same resource (see task DependentSetUp).

Fig. 7: Pareto front approximation (Makespan, Quality and Energy Cost) while cooking recipes from Table I
Fig. 8: Pareto front approximation (Makespan, Quality and Energy Cost) while cooking scaled recipes from Table I in a commercial scale scenario

The trade-off between three conflicting objectives, makespan, energy cost and deficiency (the reverse of quality), has been investigated for the example scenario and visualised in Figure 7 in a form of a Pareto front approximation. The chart demonstrates that the decreasing of makespan is obtained via sacrificing the quality (i.e., increasing deficiency). Note, during the experiment, we noticed that it is not necessary that a higher makespan must lead to a lower deficiency value (which indicates a higher food quality). This is because the recipes can be executed in parallel (based on the allocation decision during optimisation). Given the same recipes, executing in parallel can decrease the makespan but the deficiency objective will remain unchanged. Therefore, during this evaluation, we often observed new optimisation results that contain makespan and deficiency values that are both lower than certain previous optimisation results. Such a phenomenon also makes the final number of optimisation results low, as optimisation results that are strictly dominated will be removed from the Pareto front approximation. However, in general, we demonstrate the trend where the deficiency metric value is decreased (i.e., better quality is obtained) while the makespan is increasing. In addition, The trade-off between makespan and energy cost, has been investigated for the example scenario and demonstrated in Figure 7 in the form of a Pareto front approximation. The chart indicates that the decreasing of makespan can be achieved via consuming more energy.

The proposed method is applicable rather to large restaurants or company cafeterias where trade-offs between conflicting objectives such as energy, makespan and deficiency, needs to be investigated rather than to single hobs treated as home appliances. With a larger scenario, both in terms of the number of available resources and the ordered food, it needs to be considered to demonstrate the scalability of the proposed approach. Hence in the second scenario, the presence of 4 hobs from Figure 4 is assumed. In addition, the amount of the ordered food is 4 times larger than in the previous case.

Food type Recipe name Predecessor Amount (g) Cooking zone Pot type Energy (kJ) Cooking time (min) Monetary cost (€) Deficiency
Boiled water A - 1000 Hob(1), Hob(2), Hob(3), Hob(4) Pot 1 350 15 0.03 5
B - 2000 Hob(5), Hob(6 Pot 2 1400 10 0.12 8
C - 3000 Hob(7) Pot 3 3150 5 0.27 11
Pasta A Boiled water A 100 Hob(1), Hob(2), Hob(3), Hob(4) Pot 1 840 30 0.021 2
B Boiled water A 100 Hob(1), Hob(2), Hob(3), Hob(4) Pot 1 770 25 0.018 9
C Boiled water B 200 Hob(5), Hob(6) Pot 2 1120 20 0.021 14
D Boiled water B 200 Hob(5), Hob(6) Pot 2 1190 15 0.018 19
E Boiled water C 300 Hob(7) Pot 3 1520 10 0.021 22
F Boiled water C 300 Hob(7) Pot 3 1590 5 0.018 25
Rice A Boiled water A 200 Hob(1), Hob(2), Hob(3), Hob(4) Pot 1 1260 50 0.045 7
B Boiled water A 200 Hob(1), Hob(2), Hob(3), Hob(4) Pot 1 1400 45 0.039 15
C Boiled water B 400 Hob(5), Hob(6) Pot 2 1610 40 0.045 19
D Boiled water B 400 Hob(5), Hob(6) Pot 2 1750 35 0.039 22
E Boiled water C 600 Hob(7) Pot 3 1960 15 0.045 28
F Boiled water C 600 Hob(7) Pot 3 2100 13 0.039 33
Meat (beef) A Boiled water A 250 Hob(1), Hob(2), Hob(3), Hob(4) Pot 1 4550 120 0.27 5
B Boiled water A 250 Hob(1), Hob(2), Hob(3), Hob(4) Pot 1 6650 110 0.18 9
C Boiled water B 500 Hob(5), Hob(6) Pot 2 6900 90 0.27 12
D Boiled water B 500 Hob(5), Hob(6) Pot 2 7000 85 0.18 16
E Boiled water C 750 Hob(7) Pot 3 7350 60 0.27 21
F Boiled water C 750 Hob(7) Pot 3 7550 55 0.18 27
Vegetable A Boiled water A 200 Hob(1), Hob(2), Hob(3), Hob(4) Pot 1 1750 42 0.066 3
(potatos) B Boiled water A 200 Hob(1), Hob(2), Hob(3), Hob(4) Pot 1 1890 40 0.06 11
C Boiled water B 400 Hob(5), Hob(6) Pot 2 2100 32 0.066 19
D Boiled water B 400 Hob(5), Hob(6) Pot 2 2240 30 0.06 23
E Boiled water C 600 Hob(7) Pot 3 2450 22 0.066 26
F Boiled water C 600 Hob(7) Pot 3 2590 20 0.06 31
Cooking A - 200 Hob(1), Hob(2), Hob(3), Hob(4) Pot 1 700 38 0.072 11
with oil B - 200 Hob(1), Hob(2), Hob(3), Hob(4) Pot 1 840 36 0.06 16
(mushroom) C - 300 Hob(5), Hob(6) Pot 2 910 25 0.09 19
D - 300 Hob(5), Hob(6) Pot 2 1050 23 0.078 20
E - 400 Hob(7) Pot 3 1120 12 0.108 26
F - 400 Hob(7) Pot 3 1260 10 0.096 29
TABLE I: Parameters of food cooking using various cooking zones, pots and recipes

V Concluding remark

In this paper, a real-world food cooking planning and scheduling problem in a commercial smart kitchen has been described. Its goal is not only to minimise the cooking time but also to minimise the energy dissipation and maximising the food quality via selecting recipes multisubset to be executed. A typical multi-objective genetic algorithms named MOEA/D has been used. The experiments have demonstrated the applicability of the proposed approach which has been able to determine the trade-offs between the conflicting objectives. The proposed algorithm is scalable enough to be applied to a relatively large kitchen and high quantity of production.


The authors acknowledge the support of the EU H2020 SAFIRE project (Ref. 723634).

The authors would like to thank Andrea De Angelis and Claudio Cenedese from ELECTROLUX AS for their support in defining the scenario described in this paper.

The icons used in this paper have been created by Icons8,


  • [1] S. Wang, J. Wan, D. Zhang, D. Li, and C. Zhang, “Towards smart factory for industry 4.0: a self-organized multi-agent system with big data based feedback and coordination,” Computer Networks, vol. 101, pp. 158 – 168, 2016, industrial Technologies and Applications for the Internet of Things.
  • [2] R. Murray-Gibbons and C. Gibbons, “Occupational stress in the chef profession,” International Journal of Contemporary Hospitality Management, vol. 19, 02 2007.
  • [3] M. Mizrahi, A. Golan, A. B. Mizrahi, R. Gruber, A. Z. Lachnise, and A. Zoran, “Digital gastronomy: Methods & recipes for hybrid cooking,” in Proceedings of the 29th Annual Symposium on User Interface Software and Technology, ser. UIST ’16.   New York, NY, USA: ACM, 2016, pp. 541–552. [Online]. Available:
  • [4] J. Kiesel. (2018) The internet of things in restaurants. [Online]. Available:
  • [5] M. Podrazka, E. Baczynska, M. Kundys, P. S. Jelen, and E. Witkowska Nery, “Electronic tongue—a tool for all tastes?” Biosensors, vol. 8, no. 1, 2018. [Online]. Available:
  • [6] A. Sato, K. Watanabe, and J. Rekimoto, “Mimicook: A cooking assistant system with situated guidance,” in Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction, ser. TEI ’14.   New York, NY, USA: ACM, 2013, pp. 121–124. [Online]. Available:
  • [7] J. Lei, X. Ren, and D. Fox, “Fine-grained kitchen activity recognition using rgb-d,” in Proceedings of the 2012 ACM Conference on Ubiquitous Computing, ser. UbiComp ’12.   New York, NY, USA: ACM, 2012, pp. 208–211. [Online]. Available:
  • [8] Electrolux. (2018) Electrolux connected steam oven with built-in camera makes its market debut (press release). [Online]. Available:
  • [9] Y. Suzuki, H. Shinkou, and H. Ueda, “Influences of a robot’s presence and speeches in a cooking support system,” in Proceedings of the ACM Multimedia 2012 Workshop on Multimedia for Cooking and Eating Activities, ser. CEA ’12.   New York, NY, USA: ACM, 2012, pp. 31–36. [Online]. Available:
  • [10] P. Angara, M. Jiménez, K. Agarwal, H. Jain, R. Jain, U. Stege, S. Ganti, H. A. Müller, and J. W. Ng, “Foodie fooderson a conversational agent for the smart kitchen,” in Proceedings of the 27th Annual International Conference on Computer Science and Software Engineering, ser. CASCON ’17.   Riverton, NJ, USA: IBM Corp., 2017, pp. 247–253. [Online]. Available:
  • [11] R. Hamada, J. Okabe, I. Ide, S. Satoh, S. Sakai, and H. Tanaka, “Cooking navi: Assistant for daily cooking in kitchen,” in Proceedings of the 13th Annual ACM International Conference on Multimedia, ser. MULTIMEDIA ’05.   New York, NY, USA: ACM, 2005, pp. 371–374. [Online]. Available:
  • [12] P. Dziurzanski, S. Zhao, J. Swan, L. S. Indrusiak, S. Scholze, and K. Krone, “Solving the multi-objective flexible job-shop scheduling problem with alternative recipes for a chemical production process,” in

    Applications of Evolutionary Computation

    .   Springer International Publishing, 2019, pp. 33–48.
  • [13] P. Dziurzanski, J. Swan, L. S. Indrusiak, and J. Ramos, “Implementing digital twins of smart factories with interval algebra,” in IEEE International Conference on Industrial Technology, ser. ICIT 2019 2019, 2019.
  • [14] J. Wan, S. Tang, D. Li, M. Imran, C. Zhang, C. Liu, and Z. Pang, “Reconfigurable smart factory for drug packing in healthcare industry 4.0,” IEEE Transactions on Industrial Informatics, vol. 15, no. 1, pp. 507–516, Jan 2019.
  • [15] O. Sobeyko and L. Mönch, “Integrated process planning and scheduling for large-scale flexible job shops using metaheuristics,” International Journal of Production Research, vol. 55, no. 2, pp. 392–409, 2017. [Online]. Available:
  • [16] S. Zhao, P. Dziurzanski, M. Przewozniczek, M. Komarnicki, and L. S. Indrusiak, “Cloud-based dynamic distributed optimisation of integrated process planning and scheduling in smart factories,” in Proceedings of the Genetic and Evolutionary Computation Conference, ser. GECCO ’19.   New York, NY, USA: ACM, 2019.
  • [17] R. Geyer, “Identifying workloads for the cloud,” 2012. [Online]. Available:
  • [18] Y. Alsafi and V. Vyatkin, “Ontology-based reconfiguration agent for intelligent mechatronic systems in flexible manufacturing,” Robotics and Computer-Integrated Manufacturing, vol. 26, no. 4, pp. 381 – 391, 2010.
  • [19]

    Q. Zhang and H. Li, “MOEA/D: A multiobjective evolutionary algorithm based on decomposition,”

    IEEE Trans. on Evolutionary Computation, vol. 11, no. 6, pp. 712–731, Dec 2007.