The High Cost of High Tech: The Financial Dilemma GenAI Poses For Entrepreneurs

While artificial intelligence has been decades in the making, only recently has the tech industry generated such high expectations for how it will usher in a new era for digital innovation.

Following the AI boom in 2023, the spotlight has been even more on this technology, ensuring that this would be the year when companies went all-in on AI. 

However, for both startups and enterprises, recognizing the value AI provides for the bottom line still looks a ways away. 

In fact, tech leaders have reportedly been giving talks to team members to calm fears that they have been caught in a hype cycle that will not pay off.

Part of the reason this value is taking longer than expected to emerge is due to operating costs. From computer chips to cloud computing infrastructure, the components required to run big data and AI analytics projects come at a high price, for both startups and larger enterprises.

Moreover, these costs increase exponentially with the size of the data sets that algorithms process. 

Recently, SQream, a New York-based data analytics acceleration company, released the results of a landmark survey analyzing the latest trends in analytics and AI. 

Its report, titled “2024 State of Big Data Analytics: Constant Compromising Is Leading to Suboptimal Results,” highlights how widespread the problem of cost control is becoming, how this can jeopardize AI projects, and what actions can be taken to help solve these challenges.

Rising costs threaten AI projects

From intelligent product development to data-driven decision-making, it’s no wonder entrepreneurs have turned to AI to drive business growth.

Yet, the new report from SQream reveals that, in reality, these gains are proving extremely costly to achieve. The findings are based on a survey of 300 data management professionals from U.S. companies with at least $5 million in annual cloud and infrastructure spending.

Despite the generous budget allocated to this area of business operations, unfortunately costs for entrepreneurs and executives continue to rise. Seventy-one percent of respondents – more than 2 out of 3 companies – stated that they were surprised by the costs of their cloud analytics bill.

Meanwhile, 98% of respondents indicated that they had experienced a project failure by 2023, not an encouraging figure for companies planning to invest in such projects. 

As more systems move to the cloud, operational processes are now largely digitized, and enterprise organizations are left trying to manage immense data sets that are increasingly costly to store and maintain. In addition, individual AI queries become more expensive due to the computational power required to process these mega-volumes of data. 

As a result, many entrepreneurs and executives reported that their AI projects are often compromised. The biggest factor contributing to project failure in 2023 was a lack of budget (29%). Meanwhile, 41% of companies consider the high costs involved in ML experimentation to be the main challenge they face. 

Herein lies the crux of the real barrier to AI adoption that companies must overcome for projects to be sustainable.

Until now, the most common way to improve the power of big data projects was to include more CPUs. However, according to the report, this strategy is one of the main reasons that generate unsustainable costs and problems for entrepreneurs today.

For business leaders, common data management practices are not adequate to manage the current magnitude of data in 2024. 65% of the surveyed enterprise organizations employ 3 to 4 tools to perform data science tasks. Forty-two percent employ 3 to 4 tools for data processing and another 46% use 4 tools for business intelligence.

The SQream report indicates that the key problem lies in the number of tools used on average. Using multiple tools means that there is generally no single source of truth. It also increases the likelihood that bottlenecks occur in the sequence of tasks that slow down the speed of innovation.

The survey suggests that GPUs are a source of optimism for both companies and the future of AI. According to 75% of respondents, the addition of GPU instances will have the biggest impact on their data analytics and AI and ML goals by 2024.

According to SQream’s Chief Revenue Officer Deborah Leff, leaders and entrepreneurs are increasingly aware of the transformative power of GPU acceleration. The huge benefit of significant efficiency gains is too important to overlook in the competition to be an AI-driven company.

GPUs can increase processing speed and play an important role in closing the gap between the high costs and the benefits to enterprises.