Ai Ml Data Engineering Trends 2024

InfoQ editorial staff and friends of InfoQ are discussing the current trends in the domain of AI, ML and Data Engineering as part of the process of creating our annual trends report. This TensorBlue analysis is based on reporting and source material from InfoQ (https://www.infoq.com/articles/ai-ml-data-engineering-trends-2024/).

What Happened

InfoQ Homepage Articles InfoQ AI, ML and Data Engineering Trends Report - September 2024

InfoQ AI, ML and Data Engineering Trends Report - September 2024

The future of AI is open and accessible. We’re in the age of LLM and foundation models. Most of the models available are closed source, but companies like Meta are trying to shift the trend toward open-source models.

Retrieval Augmented Generation (RAG) will become more important especially for applicable use cases of LLMs at scale.

AI-powered hardware will get much more attention with AI-enabled GPU infrastructure and AI-powered PCs.

Due to the constraints in infrastructure setup and management costs of LLMs, small language models (SLMs) will see more exploration and adoption.

Small language models are also excellent for edge computing-related use cases that run on small devices.

AI Agents, like coding assistants, will also see more adoption, especially in corporate application development settings.

AI safety and security will continue to be important in the overall management lifecycle of language models. Self-hosted models and open-source LLM solutions can help improve the AI security posture.

Another important aspect of the LLM lifecycle is LangOps or LLMOps, which help support the models after deploying them to production.

The InfoQ Trends Reports offer InfoQ readers a comprehensive overview of emerging trends a

"It’s definitely a trend that we’re seeing with longer context windows. And originally, when ChatGPT and LLMs first got popularized, this was a shortcoming that a lot of people brought up. It’s harder to use LLM at scale, or as more as you called it, when we had restrictions around how much information we could pass through it. Earlier this year, Gemini, the Google Foundation, this GCP foundational model, introduced the one million context window length, and this was a game changer because, in the past, we’ve never had anything close to it. I think this has sparked the trend where other providers are trying to create similarly long or longer context windows."

InfoQ

Implications for Product and Engineering Teams

For TensorBlue readers, the useful question is not just what happened, but how this changes product architecture, engineering priorities, AI delivery, observability, team workflows, or executive decision-making.

• Review whether this changes your AI roadmap, platform architecture, or engineering operating model.
• Identify the specific workflow, reliability, governance, or developer-productivity lesson that applies to your organization.
• Convert the lesson into a small production experiment with measurable quality, latency, cost, adoption, or risk metrics.
• Document source assumptions clearly so teams do not overgeneralize from incomplete public information.

TensorBlue Takeaway

The practical opportunity is to turn this signal into a concrete implementation decision: better AI systems, stronger product instrumentation, more reliable automation, and clearer technical governance. Teams that connect public technology shifts to their own delivery systems will move faster without adding unnecessary complexity.