Senior Prompt Engineer (AI / LLM)
Job description
We are looking for a Senior Prompt Engineer (M/F) for our client in the banking sector. In this role, you will shape and optimize the behaviour of AI agents used in conversational AI applications. You will focus on prompt development, agent configuration and systematic evaluation of LLM-based solutions to ensure reliable, accurate and consistent outputs.
Responsibilities
Design, develop and maintain system prompts, agent instructions and reusable prompt components.
Configure AI agents, including routing, handover logic, knowledge sources and tool usage.
Optimize LLM behaviour to ensure accurate, consistent and contextually relevant responses.
Define how AI agents handle ambiguous requests, missing information and situations beyond their capabilities.
Design evaluation datasets, test scenarios and scoring criteria to measure response quality.
Analyze conversation traces and evaluation results to identify issues and improve agent performance.
Test and compare prompt and model changes to measure their impact on output quality.
Develop regression test cases to ensure consistent agent behaviour over time.
Collaborate with engineering, product and business teams to translate requirements into reliable AI solutions.
Document prompt changes, experiments and evaluation results.
Requirements
Hands-on experience with prompt engineering for LLM-based applications.
Experience configuring and optimizing AI agents.
Strong understanding of system prompts, context management and few-shot prompting.
Experience with LLM evaluation and systematic testing of AI-generated outputs.
Understanding of tool calling, retrieval-augmented generation (RAG) and structured outputs.
Ability to analyze LLM responses, identify failure patterns and implement improvements.
Experience working with API requests, tool results and conversation traces.
Professional working proficiency in English.
Nice to Have
Experience with LangChain, LangGraph or similar agent frameworks.
Knowledge of Python scripting and Git.
Experience with automated evaluation frameworks and LLM-as-a-Judge approaches.
Familiarity with JSON schemas and structured data outputs.
Experience testing multilingual AI applications.
Previous experience in banking, financial services or other regulated environments.
Benefits
Engaging projects across diverse industries, offering continuous professional challenges and growth opportunities
Tailor-made learning & development program, complemented by monthly in-house workshops to boost your skills
Flexible working hours and the possibility of home office
Modern, inspiring office environment with daily perks such as breakfast, fresh fruit, and yoga sessions
MultiSport card to support an active lifestyle and access to a wide range of sports and wellness facilities
Company description
Who we are
Ableneo is a partner for AI and technology-driven transformation. We combine deep software engineering with business consulting to help organisations implement intelligent systems that deliver measurable results and reduce the risk of failed innovation.
What we do
• AI & Data Enabling. We deploy chatbots and LLM solutions, automate documents and workflows, and work with Computer Vision. Our focus is on solving concrete business problems with AI and achieving a high success rate in real use cases.
• Business Optimisation & Efficiency. We streamline processes, reduce delays, automate operational work, and set up more efficient ways of working.
• Product Design & Development. We design and build digital products, customer journeys, and scalable platforms that support long-term growth.
• Software Engineering. We offer flexible delivery models. From specialised roles to full teams, supporting both fast-growing companies and established enterprises.
Why join us
We have years of experience and strong references across startups, scale-ups, and large organisations. We work on assignments where AI makes real impact. For example, in the banking sector we validated an LLM solution with a high success rate in a real proof-of-concept.
At Ableneo you will work on projects that push the boundaries of AI and software engineering. You will solve complex problems, collaborate with decision-makers, and help build systems that have immediate business value.
If you want to shape the next generation of intelligent software, let’s talk.