top of page
  • Ben Tytonovich

Enterprise Infrastructure in 2024 - Thoughts & Trends

For founders and investors trying to wrap their head around main focus points for 2024 in AI delivery, data infrastructure, devops and cyber - following are some thoughts & potential trends in a (relatively) quick digest.


AI Delivery

  1. Maturity in AI delivery will have to step up as organizations will tire from more and more LLM-based PoCs failing to provide meaningful business value. The criteria for experimentation will increase and the seriousness of success criteria will be more evident for every LLM project. More resources will be diverted into adopting 3rd party LLM infrastructure solutions as startups and incumbents continue to improve their offerings while in-house development faces the harsh reality that there are no silver bullets when it comes to software development, even when LLMs are involved.

  2. Specialized API-based LLM Consumption (or LLM-less, the serverless equivalent) will become more of a norm as AI vendors will provide use case specific optimized (functionally and operationally) LLM offerings in a serverless manner for vertical use cases. This will accelerate the inclusion of more LLM-based features in software products from companies who most of us don’t necessarily consider as early adopters of AI-first technologies.

  3. AI Replacing UI is a massive gradual trend which will start to take place in 2024 in a meaningful way, where more workflows which required an employee controlling a mouse and keyboard in the past, will now be replaced by either autonomous software and/or by chat-based interfaces. As with every software trend, this will start in semi-autonomous adoptions, augmenting employees at first. But as agent-like workflows mature, this will disrupt many work processes in almost every single organizational department.

  4. AI Synthesis, which is basically the act of leveraging multi-modal model functionality in order to fuse together insights based on various modalities (whether in format - video, text & audio or in type - language, actions & time), will start taking place in several commercial use cases in a meaningful way for the first time.

  5. RAG, fine tuning and whichever evolution these processes will go through in 2024 (but essentially - the concept of AI enterprise-specific augmentation) will continue to dominate in AI delivery as the need for enterprise-grade LLM-based solutions that are precise, consistent and truthful will become the end-goal for many.

  6. Privacy and security have been marked as concerns in the midst of the LLM explosion from (almost) day one. Yet, the toolset with regards to how to put in the right guardrails with the right approach has largely been missing in 2023. The understanding of privacy and security implications (in parallel to the evolution of regulation) will mature in 2024. Presumably, with a larger portion of LLM projects heading to 3rd party providers (see point #1), some of the burden will fall on the vendors’ shoulders, who will benefit from more horizontal experience while working with several customers which will benefit the rapid evolution of best practices.

  7. The continuous commoditization of large models is a natural process for which we started to see initial signs already in 2023 and one can only assume will continue to occur in 2024. This is the result of multiple factors, including open source propositions becoming more mature. Will we eventually see a dynamic in which 95% of large model related use cases will require commoditized (and smaller?) model versions and only 5% of use cases will necessitate specialized offerings?


Data Infrastructure

  1. The unstructured data phenomenon will continue to explode as organizations identify further opportunities to leverage video, free text, raw documents, audio and various other formats. Early adopters will move into more complex use cases and newcomers will start extracting the initial value hiding in internal corporate knowledge, customer engagements, financial documents, video calls and whatnot. Needless to say, this is also the backbone for the AI chatbot trend of the past year.

  2. The whole data-in-motion category will continue to spread into more use cases and in a broader range of companies. Growing demand for immediate insights will drive adoption of real-time analytics. Machine learning processes reliant on streaming data will also naturally conquer more ground and thus increase data-in-motion’s importance. While costly and complex to implement for now, hopefully new solutions will mature to ease real-time pipeline adoption by more organizations.

  3. LLM-enhanced data features will become the norm in several existing data product categories. The low hanging fruit candidates for this are data quality and integrity platforms but also ETL-related processes, data cataloging and some data automations where LLM assistance can move the needle.

  4. Cost-awareness with regards to the various organizational data pipelines will continue to increase as IT/devops/data engineering leaders are becoming more wary of budgets dedicated toward lakehouses, infrastructure observability platforms, SIEMs and other significant data products.

  5. BI complexity, which is the result of the explosion of BI in so many companies in the past few years, will drive several new and new-ish trends to try and decrease it as much as possible. The semantic issue, involving the lack of consistency in KPI definitions (which led to the race after the illusive semantic layer) will continue to trouble organizations. The explosion of reporting dashboards in many organizations, which becomes worse and worse with every year, will also become even more troublesome. Democratization of reporting, led by conversational BI (i.e. bot-assisted reporting) will hopefully decrease some of this burden, but unfortunately, probably not by much, at least in 2024.

  6. Standard software development practices in data operations (i.e. Data as Code), where data practitioners and processes adopt solutions and best practices helping them improve operational excellence, will have to continue to take place on multiple levels, otherwise major data ops issues (data pipeline breakage, etc.) will become even more rampant as data-based processes continue to dominate the modern organization.


Devops

  1. AI-led intelligent automations with an added emphasis on autonomous decision making will likely attract more attention and traction as solutions in the field will reach maturity levels they haven’t reached before. Devops processes are a natural candidate as a target for the big AI agent question as founders will try to figure out what exactly are AI agents and how can they support devops processes in a more autonomous way. Prime candidates for autonomous automation are processes that have already gone through initial automation, such as different workflows within the CI/CD pipeline and many IaC-centric automations.

  2. Self-sufficient engineering, and specifically platform engineering as the main trend under this umbrella, will continue to spread, through internal developer portals and other methods which will lower the barrier to entry for developers to control more devops processes themselves, without significant outside help, in order to bring about more agility into R&D workflows.

  3. Existing trends from the past few years like the adoption of multi-cloud/hybrid environments will naturally continue to attract more and more attention as the need to support complex environments as a fundamental approach will continue to take place. IaC and Gitops are obviously here to stay and to evolve, as is the continuous dominance of Kuberenetes (vis-a-vis making it more automated and simpler).

  4. Vertical cloud propositions will continue to gain further traction, whether in the sector-specific sense (healthcare, bio, etc.) or the technological-specific sense (ML-related cloud workloads) and will present one of the only avenues where founders can meaningfully try and disrupt niches within the (almost) impenetrable incumbent cloud ecosystem.


Cyber Security

  1. LLM-related security threats can be divided into two main categories, at least for now - GenAI at the service of various social engineering attacks is one such category and threats on LLM infrastructure and applications is the other category. Attackers will surely leverage LLMs to enhance phishing/vishing and other techniques with more credible, multi-modal and relevant content (while potentially also fusing in more personal information), making these attacks less distinguishable. Threats on the LLM infrastructure layer (e.g. training data stored in the data warehouse) or the application layer (e.g. model manipulation / injection) are also just a matter of time. Actual CISO prioritization for 2024 for this new generation of LLM-related attacks will naturally depend on the actual threats experienced, and not just the hypotheticals.

  2. Non-human identities will continue to expand their gap on human identities, trending toward the 1 to 50 ratio of human to non-human identities per organization. This proliferation, which is driven by numerous digital transformation related processes, most of which are detailed in other parts of this article (including IaC, LLMs, RPAs, platform engineering, microservices and whatnot) will continue to aggravate the inability of security professionals to get a true grasp of how widespread non-human identities’ presence truly is. On the human side of things, the passwordless trend will continue to pick up steam as organizations have more advantages in adopting it versus potential friction.

  3. Significant productivity gains for SOC teams led by the implementation of autonomous AI is a large question mark which it shares with other infrastructure operations departments (e.g. devops, data engineering, etc.). There are certainly several use cases in which SOC teams can leverage next-gen SOAR platforms to bridge the growing pain of hard-to-find skilled security professionals. MSPs/MSSPs are also a natural candidate to benefit from a potential autonomous software boost to their operations, as the cyber security professionals shortage will continue to increase their traction.

  4. Devsec’s integration into every facet of the software development lifecycle will also take another step into maturity, with further consolidation of devsec offerings, more AI-reliant solutions decreasing the friction of adding them into dev workflows and additional companies making continuous security testing and vulnerability scanning standard practices. This is inevitable with the growth of software supply chain based continuous rise and even the potential of more code-based vulnerabilities being introduced due to the exponential growth of auto code generation being only semi-supervised.

Recent Posts

See All

Trunks and Branches

One of the key metaphors I’ve been using in recent years is equating the common software stack into a tree trunk and branches. For most personas/departments, there is one main piece of software that d

The Maple Approach

I’ve been in the VC business on and off since 2010. Surprisingly enough, it could be said that our business has remarkably changed since and it could also be said that it hasn’t changed all that much.

Comments


bottom of page