Challenges: 

• Data Variety and Unstructured Nature: The insurance client encountered data from various sources, often unstructured, making it challenging to extract meaningful insights. 
• Multi-Source Decision-Making: Businesses typically relied on an average of five data sources to make informed decisions, highlighting the intricacies of their data landscape. 
• Unstructured Data Overload: According to IDC, 80% of data is unstructured, posing a significant challenge for traditional data warehouses.
• AI and ML Technology Adoption: With artificial intelligence becoming a core enterprise capability, industry analysts report that over 80% of organizations are expected to adopt AI-powered solutions by 2026, accelerating the demand for advanced analytics, automation, and intelligent decision-making platforms. 
• Exponential Data Growth: Global data creation continues to surge rapidly, with analysts predicting that global data volumes will exceed 180 zettabytes by 2025, driven by billions of connected devices and over 6 billion consumers interacting with digital data daily. This explosive growth requires scalable data platforms and high-performance processing solutions. 
• Demand for Faster Data Processing: Businesses demanded faster access, processing, and analysis of data for innovation, competitive advantage, efficiency, and compliance. 

Solutions: 

Enkaytech proposed a transformative solution for the insurance client by leveraging a cloud data lake house. This innovative approach combined the strengths of a data lake and a data warehouse, providing a comprehensive platform for processing diverse enterprise and streaming data for business analysis and machine learning.

Key Features of the Data Lake House Architecture: 

• Operational Efficiency: Seamlessly use data without replication across the data lake and warehouse.

• Resource Optimization: Decouple storage and compute resources, consuming only what is needed at any given time.

• Diverse Data Support: Enhanced multi-model and polyglot architecture supporting various data types.

• Versatile Use Cases: Support for streaming analytics, business analytics, data science, and machine learning.

• Fabric Data Agent Integration: In addition to the core Lakehouse architecture, we Implemented a Microsoft Fabric Data Agent (powered by generative AI and Azure OpenAI) to enable a natural language interface for querying and analyzing data across the Lakehouse, Warehouses, and Power BI semantic models. 

This conversational AI layer allows business users and analysts to ask questions in plain English (e.g., fraud indicators in claims or customer sentiment trends) and receive accurate insights with visualizations, tables, or narratives—without writing SQL, DAX, or KQL. 

Using NL2SQL and NL2DAX, the agent translates natural language into secure queries, delivering human-readable insights while maintaining row-level security, governance, and compliance. 

Implementation Road Map: 

Enkaytech initiated the transformation of the insurance provider’s Azure Data Lake infrastructure into a platform capable of leveraging AI and analytics. The roadmap included integrating various Azure services to enhance data processing capabilities:  

– Azure Event Hubs 

– Azure Synapse Analytics 

– Azure Data Lake Gen2 

– Azure Cognitive Services 

– Azure IoT Hub 

– SQL Data Warehouse 

– Azure Purview 

– Azure Stream Analytics 

– Azure Machine Learning 

– Azure Data Factory 

– Microsoft Power BI 

– Azure DevOps 

– Azure Active Directory 

– GitHub 

• Microsoft Fabric Platform Extension: Evolving the Azure Data Lake infrastructure further into Microsoft Fabric, incorporating: 
• Fabric Data Agent (preview/GA features) for building custom conversational Q&A experiences over OneLake data. 
• Integration with Copilot in Fabric for enhanced natural language capabilities in notebooks, pipelines, reports, and real-time dashboards. 
• Connection to existing services like Azure Synapse Analytics, Azure Data Lake Gen2, Power BI, and Azure Machine Learning for unified, governed access. 
• Phased rollout: Configure data sources → Customize agent instructions, examples, and business context → Test and evaluate agent responses → Embed/share via Microsoft Copilot Studio, Power BI, Teams, or custom applications. 

Advanced AI Capabilities (Fabric Data Agent) 

Overview: Building on the data lakehouse foundation, the Fabric Data Agent introduces generative AI-driven conversational analytics, making the insurance client’s unified data platform truly accessible to every user. 

Key Advantages: 

• Zero-code querying for business users. 
• Accurate, governed responses grounded in enterprise data (no hallucinations via schema-aware processing). 
• Multi-source support (lakehouse tables, warehouses, semantic models). 
• Scalable to streaming/real-time scenarios when combined with Azure Stream Analytics or Real-Time Intelligence in Fabric. 

Business Use Cases/Benefits:

• AI Model Reuse: Reusing AI models built on the data lake for multiple applications, enhancing operational efficiency.
• Text Extraction from Images: Converting images of forms or driver’s licenses to text for easy accessibility via APIs, benefiting various internal applications. 
• Fraud Detection: Utilizing advanced analytics for effective fraud detection within the enterprise. 
• Personalized Price Quotes: Analyzing customer characteristics to generate personalized price quotes, enhancing customer engagement.
• Sentiment Analysis for Customer Care: Predicting customer churn/attrition through sentiment analysis of customer care responses.
• APIs for Data Lake: Building APIs to the data lake and replacing Operational Data Stores (ODS) for streamlined data access.
• Natural Language Data Interaction (Conversational Analytics): Empowers non-technical users across the insurance company (e.g., claims adjusters, agents, executives) to self-serve. 
• Democratized Access to Insights: Business users can query unstructured-enriched data (e.g., sentiment from customer emails/voice transcripts, OCR-extracted form data, fraud patterns from claims videos) in everyday language.
• Seamless Integration & Consumption: The Fabric Data Agent can be integrated into Microsoft Fabric workspaces and Power BI reports, embedded into custom AI agents via Copilot Studio (e.g., Teams or chatbots), and connected through APIs or low-code integrations for internal apps and portals—enabling governed, conversational access to data. 

Conclusion 

Enkaytech’s implementation of the cloud data lake house solution empowered the large insurance organization to overcome its data challenges, providing a foundation for enhanced business intelligence, innovation, and competitive advantage. The collaborative effort showcased the potential of advanced data processing technologies in addressing complex business requirements.