Overall target
With the help of LLM-supported chatbots, support, human resources, tenders and general company knowledge are to be supported. The efficiency of employees is to be increased. Specialist staff can be supported. Knowledge of employees who leave the company should be retained. A high level of data and information security should be guaranteed through self-hosting. In addition, the use of small LLMs should minimise resource requirements.
The rapid development of large language models (LLMs) offers small and medium-sized enterprises (SMEs) new opportunities to automate processes and increase efficiency. The concept of fine-tuning LLMs on regional infrastructure is particularly exciting, allowing companies to retain greater independence from large US cloud providers. In this context, it is not only important to maintain data protection standards, but also to implement sustainable and resource-efficient IT solutions.
Social and legal objectives
Promotion of data protection and data security
A key objective is to develop a solution that meets the highest data protection requirements and preserves data sovereignty for companies. A high level of information security is achieved through independence from global cloud services, which creates trust among users and strengthens Baden-Württemberg as a pioneer in data protection-compliant digitalisation.
Sustainability and resource conservation
The project aims to build a resource-efficient IT infrastructure that reduces the carbon footprint and contributes to climate protection goals.
State of the art
LLMs are usually operated via international cloud providers such as AWS, Microsoft Azure or Google Cloud, which raises data protection and sustainability concerns. However, open source alternatives such as Llama, Gemma and Mistral offer the option of operating these models locally and in compliance with data protection regulations, which is particularly advantageous for small and medium-sized enterprises (SMEs). They can be customised through fine-tuning and can therefore be used in a resource-saving manner.
With an increasing focus on sustainability, LLMs should be designed to be more energy-efficient in order to reduce their carbon footprint. Green IT approaches and specialised models are being researched in order to optimise computing resources. At the same time, there is still a need for innovation, particularly in the area of data protection-compliant implementation and the development of customised, resource-saving models that do not require large cloud services. Such developments would allow European SMEs to utilise powerful LLMs while complying with data protection and sustainability standards, enabling broader applications in operational processes.
More information on the state of the art
Previews of the chatbot
And if you want to get started right away, you can find the first previews of the chatbot under Examples.
