With an increasing share of distributed energy resources (DER) in the electrical energy system it is becoming more and more important that DER not only take part in active power provision but are also involved in the provision of ancillary services like frequency control or voltage regulation. Due to the large number of DER connected to the lower voltage levels via power-electronic converters the distribution grid evolves from a formerly mostly passive to a highly active system with a high number of actuating variables distributed over multiple stakeholders. The coordination and optimization of this kind of distribution grid requires new control and optimization approaches, not only with regard to the distribution grid itself, but also with regard to the coordination with the overlying transmission grid. This abstract presents first ideas of a PhD-project that aims to use machine learning surrogate models and decoder functions for agent-based dynamic optimization of local controller configurations particularly with regard to voltage regulation. Decoder functions derived from machine learning surrogate models abstract optimization problems from technical system specifications and allow for constraint-free optimization with standard optimization heuristics such as evolutionary optimization methods.