OpenAI Appoints AI Safety Expert Zico Kolter to Board Amid Rising Concerns

Welcome to Towards AGI, your premier newsletter dedicated to the world of Artificial Intelligence. Our mission is to guide you through the evolving realm of AI with a specific focus on Generative AI. Each issue is designed to enrich your understanding and spark your curiosity about the advancements and challenges shaping the future of AI.

Whether you're deeply embedded in the AI industry or just beginning to explore its vast potential, "Towards AGI" is crafted to provide you with comprehensive insights and discussions on the most pertinent topics. From groundbreaking research to ethical considerations, our newsletter is here to keep you at the forefront of AI innovation. Join our community of AI professionals, hobbyists, and academics as we pursue the ambitious path toward Artificial General Intelligence. Let’s embark on this journey together, exploring the rich landscape of AI through expert analysis, exclusive content, and engaging discussions.

TheGen.AI News

OpenAI Appoints AI Safety Expert Zico Kolter to Board Amid Rising Concerns

OpenAI has appointed Zico Kolter, a professor and director of the machine learning department at Carnegie Mellon University, to its board, the Microsoft-backed AI startup announced on Thursday. The board of directors at the maker of ChatGPT has seen several changes recently due to increasing concerns over the safety of generative artificial intelligence, as more businesses rapidly adopt this emerging technology.

Kolter, who primarily focuses on AI safety, will also join OpenAI's safety and security committee, which includes CEO Sam Altman and directors Bret Taylor, Adam D'Angelo, Paul Nakasone, and Nicole Seligman, according to the company.

The safety committee, established in May, is responsible for making recommendations on safety and security decisions for all OpenAI projects. OpenAI's generative AI-powered chatbots, which can engage in human-like conversations and create images from text prompts, have raised safety concerns as these models grow more powerful.

Kolter, previously the chief data scientist at C3.ai, currently serves as the chief expert at Bosch and the chief technical adviser at Gray Swan, a startup focused on AI safety and security.

In 2023, Kolter played a key role in developing methods for automatically assessing the safety of large language models, showcasing the potential to bypass existing model safeguards, as noted in OpenAI's blog post.

In July, Microsoft relinquished its board observer seat at OpenAI, a move intended to address concerns from U.S. and UK antitrust regulators about the company's level of control over the startup as generative AI continues to gain traction.

Early Gen-AI Implementers See Strong Gains, Google-Commissioned Survey Reveals

Generative AI, despite its potential, can be costly at scale, and the return on investment (ROI) isn't always obvious, leading some businesses to hesitate in adopting the technology. However, Google, which has heavily invested in generative AI, believes that enterprises should embrace it. According to Oliver Parker, VP of global generative AI go-to-market at Google Cloud, early adopters of this technology are already benefiting from increased revenue, improved customer service, and enhanced productivity.

These insights come from a report by the National Research Group (NRG), commissioned by Google, which found that early adopters are seeing tangible revenue increases. Out of 2,508 C-suite executives surveyed, 61% had implemented generative AI tools, and 86% of those reported revenue gains exceeding six percent. Google clarified that this estimate is based on past and current generative AI initiatives.

The report highlights the positive impact of generative AI across various areas, including productivity, security, business growth, and user experience. Of the respondents, 43% noted a significant boost in productivity, with 45% of that group estimating that these tools had more than doubled employee productivity. Additionally, 39% reported business growth due to generative AI, with 77% of them citing improved lead generation and customer acquisition. User experience also improved, with 37% of respondents indicating a meaningful impact, 85% of whom saw increased engagement, and 80% reported higher customer satisfaction.

In terms of security, 56% of executives said generative AI had strengthened their organization's security, with 82% of that group citing better threat detection and 71% noting faster incident resolution.

Despite these findings, the report also reveals a significant gap in generative AI adoption, with 39% of enterprises not yet using the technology in production. This figure contrasts with a February report from the Census Bureau, which found that only 5.4% of US businesses were using AI, and even in the information sector, adoption was just 18%.

The NRG survey indicates that 47% of respondents plan to use generative AI to develop new products and services, and 49% aim to enhance profit margins with the technology.

Google is not alone in promoting the benefits of generative AI. Major cloud providers like Microsoft and AWS have invested heavily in AI infrastructure and need to convince customers of the technology's value. For example, Amazon CEO Andy Jassy claimed that the company's generative AI tool "Q" saved Amazon $260 million by transforming Java applications, though this is less than one percent of Amazon's 2023 profits.

Microsoft, meanwhile, has integrated generative AI into its products, such as GitHub Copilot and Copilot for Office 365, but potential customers remain skeptical about the added costs, despite claims of increased productivity.

While AI service providers emphasize potential ROI, some experts, like Gartner analyst Frances Karamouzis, advise against trying to calculate AI ROI, as it may not be easily reflected in financial statements.

Over 60% of Global Companies Lack Gen AI Usage Policies, Report Finds

Despite the rapid adoption of Generative Artificial Intelligence (Gen AI) across various industries, nearly 60% of companies in the procurement sector still lack specific policies governing its use, according to a new report. This suggests that many organizations are in the early stages of understanding and incorporating AI into their procurement processes. A survey by Zycus-Procurement Tactics highlighted the state of Gen AI in procurement, revealing that 69.8% of respondents do not currently use Gen AI-based software in their procurement activities.

The “State of Generative AI in Procurement” report offers insights into how Gen AI is transforming procurement processes, drawing on survey data from procurement leaders worldwide. One key finding is that nearly 70% of Chief Procurement Officers (CPOs) have yet to implement Gen AI in their operations, highlighting a significant governance challenge and underscoring the need for strategic guidance as adoption increases.

The report also explores Gen AI's transformative potential, noting its role in automating routine tasks and enhancing strategic decision-making throughout the procurement lifecycle. Interestingly, the survey found that 50.6% of respondents use Gen AI in contract management, primarily to improve administrative efficiency. Additionally, 31.5% of respondents use Gen AI to develop more effective negotiation strategies, while only 11.5% use it to streamline communication with suppliers, aligning with the personalized nature of supplier relationship management.

Data quality and security emerged as top concerns among respondents, emphasizing the importance of responsible AI implementation strategies. The report encourages organizations to adopt comprehensive risk management approaches to ensure that the advancements in Gen AI technology are supported by robust governance and security measures.

While the potential of Gen AI is vast, the adoption gap remains significant. Organizations that accelerate their implementation and integrate Gen AI responsibly are likely to gain a substantial competitive advantage.

Hiscox Unveils Generative AI-Enhanced Underwriting Platform

Hiscox, a global specialist insurer, has introduced the London insurance market’s first lead underwriting platform powered by generative AI, designed to expedite and enhance the process of providing sabotage and terrorism insurance quotes.

Originally developed as a proof of concept in December 2023 in partnership with Google Cloud, this innovative underwriting platform allows Hiscox’s sabotage and terrorism business line to dramatically reduce the time required from risk submission to quote generation.

Initially, the AI-enhanced platform will focus on renewing existing sabotage and terrorism policies in the US and Canada, excluding New York and Chicago metro areas.

By integrating Hiscox AI Laboratories (Hailo) with Google Cloud’s generative AI, the platform eliminates manual elements in the underwriting process. Hiscox first launched Hailo in 2021 as its proprietary AI tool. The platform assesses eligible risks using Google Cloud’s Gemini large language model, which generates an email for brokers with pre-filled pricing and data, ready for underwriter review. This enables Hiscox to deliver an insurance quote to brokers in just minutes.

The new platform is available to all brokers, with the first risk being underwritten in collaboration with WTW. Hiscox and Google Cloud plan to continue refining the platform’s capabilities and exploring its application in other business lines.

Kate Markham, CEO of Hiscox London Market, expressed excitement over the platform’s impact, particularly in the sabotage and terrorism sector, noting that the technology helps deliver clear benefits to customers while freeing underwriters to focus on more complex risks.

Graham Drury, head of Financial Services Industry UK at Google Cloud, highlighted the successful transformation of complex insurance underwriting processes through generative AI, and expressed a commitment to further evolving the augmented underwriting model.

Jo Holliday, global head of Crisis Management at WTW, emphasized the importance of digitization for the future of the insurance market, praising the use of AI to streamline processes and accelerate turnaround times from risk submission to quote, benefiting clients in the evolving digital marketplace.

Hiscox, headquartered in Bermuda and listed on the London Stock Exchange, employs over 3,000 people across 14 countries. The company offers a wide range of specialist insurance products through its retail businesses in the UK, Europe, Asia, and the USA, with international and large-scale business underwritten through Hiscox London Market and Hiscox Re & ILS.

TheOpensource.AI News

Apple’s ToolSandbox Shows Open-Source AI Falling Short of Proprietary Models

Apple researchers have introduced ToolSandbox, a new benchmark designed to more comprehensively evaluate the real-world capabilities of AI assistants. Published on arXiv, the research addresses significant gaps in current evaluation methods for large language models (LLMs) that utilize external tools to perform tasks.

ToolSandbox features three critical elements often missing in other benchmarks: stateful interactions, conversational abilities, and dynamic evaluation. Lead author Jiarui Lu explains that ToolSandbox incorporates stateful tool execution, implicit state dependencies between tools, a user simulator for on-policy conversational evaluation, and a dynamic evaluation strategy.

This benchmark aims to more closely mimic real-world scenarios. For example, it can test whether an AI assistant understands the need to enable a device’s cellular service before sending a text message—requiring reasoning about the system’s current state and making the necessary adjustments.

Proprietary models outperform open-source, but challenges persist

The researchers evaluated various AI models using ToolSandbox, discovering a notable performance gap between proprietary and open-source models.

This finding contradicts recent reports that suggest open-source AI is rapidly catching up to proprietary systems. Just last month, startup Galileo released a benchmark indicating that open-source models are closing the gap with leading proprietary models, and companies like Meta and Mistral announced open-source models they claim are on par with top proprietary systems.

However, the Apple study found that even the most advanced AI assistants struggled with complex tasks involving state dependencies, canonicalization (standardizing user input), and situations with insufficient information.

The authors of the paper note that both open-source and proprietary models exhibit a significant performance gap, and complex tasks such as State Dependency, Canonicalization, and Insufficient Information as defined in ToolSandbox present challenges even for the most advanced LLMs, offering new insights into the capabilities of LLMs that use tools.

Interestingly, the study revealed that larger models sometimes performed worse than smaller ones in certain scenarios, particularly those involving state dependencies, indicating that model size does not always correlate with better performance in complex, real-world tasks.

The complexity of AI performance: Size isn't everything

The introduction of ToolSandbox could have significant implications for the development and assessment of AI assistants. By providing a more realistic testing environment, it may help researchers identify and address key limitations in current AI systems, ultimately leading to more capable and reliable AI assistants for users.

As AI continues to integrate more deeply into everyday life, benchmarks like ToolSandbox will be crucial in ensuring these systems can manage the complexity and nuance of real-world interactions.

The research team has announced plans to release the ToolSandbox evaluation framework on Github soon, inviting the broader AI community to build upon and refine this important work.

While recent advances in open-source AI have generated excitement about democratizing access to cutting-edge AI tools, the Apple study serves as a reminder that significant challenges remain in developing AI systems capable of handling complex, real-world tasks.

As the field continues to evolve rapidly, rigorous benchmarks like ToolSandbox will be essential in distinguishing between hype and reality, guiding the development of truly capable AI assistants.

FLUX.1: The Open-Source AI Image Generator Set to Rival Midjourney

The latest trend in AI-generated photos taking social media by storm features close-ups of men and women speaking into microphones, seemingly at a conference or event. These images, which first appeared on a Reddit thread focused on AI-generated content, have sparked widespread attention due to their highly realistic depiction of skin, hair, and wrinkles.

The photos were created using a newly launched text-to-image AI model called FLUX.1, developed by the German company Black Forest Labs. While similar images have surfaced online before, those generated by FLUX.1 demonstrate that AI models are now capable of producing fake people with no obvious flaws, aside from the text on their badges. This indicates that the technology is becoming more advanced and accessible, as FLUX.1 is an open-source model.

It's important to note that the viral FLUX.1-generated photos were further refined using a Low-Rank Adaptation (LoRA) technique, which optimizes the outputs of large language models.

FLUX.1, launched on August 1, is available in three versions: the high-end ‘Pro’ version for commercial use, the ‘Dev’ version with open weights for non-commercial use, and the faster ‘Schnell’ version, which also has open weights and is named after the German word for "quick."

The AI model's architecture combines transformer and diffusion techniques, with a parameter count of 12 billion. For comparison, Stability Diffusion 3 models range from 800 million to 8 billion parameters. FLUX.1 has been further optimized using training methods like flow matching, according to a blog post by Black Forest Labs, though the company did not disclose the source of its training data.

Black Forest Labs, an AI startup founded by a team of researchers and engineers, including former key members of Stability AI, developed FLUX.1. Robin Rombach, Andreas Blattmann, and Dominik Lorenz were reportedly working at Stability AI until the launch of Stability Diffusion 3 (SD3).

Recently, Black Forest Labs raised over $31 million in seed funding from Silicon Valley investors like Andreessen Horowitz, General Catalyst, and MätchVC. AI researcher Matthias Bethge and former Disney president Michael Ovitz have also joined as advisers, according to ArsTechnica.

FLUX.1 appears to be capable of accurately generating human hands and legs, a challenge that has plagued previous AI models due to limitations in training datasets.

“All FLUX.1 model variants support a wide range of aspect ratios and resolutions between 0.1 and 2.0 megapixels,” the company stated. Black Forest Labs is also developing a text-to-video generator to compete with models like OpenAI’s Sora, Runway’s Gen-3 Alpha, and Kuaishou’s Kling.

The weights of FLUX.1 [dev] and FLUX.1 [schnell] are available on AI developer platforms like Hugging Face, and the models can be directly accessed through AI cloud-hosting platforms Replicate and Fal. Additionally, stock image site Freepik has integrated FLUX.1 into its AI toolbox.

Linux Foundation Leads the Charge in Open-Source AI with New Initiative

The Linux Foundation is now overseeing the Open Model Initiative (OMI), an effort dedicated to developing free and open-source large language models (LLMs) that remain accessible to everyone. Additionally, this initiative aims to establish clear ethical guidelines and shared standards.

Through the Open Model Initiative, the Linux Foundation seeks to make significant strides in offering genuinely “open” generative AI models, responding to the increasing demand for such resources.

OMI is also addressing the recent pullbacks by certain AI companies, such as OpenAI, which have limited the deployment of generative AI by altering licensing requirements. For instance, Stability AI recently imposed restrictions on its popular image generation model, Stable Diffusion 3 (SD3), which was previously available indefinitely but now requires a monthly subscription and has other usage limitations. The licensing terms for SD3 are also unclear.

OMI was founded by several organizations, including Invoke AI, CivitAI, and Comfy Org, with support from community projects like Sentient Foundation and Wand Synthesis AI. Within the Linux Foundation, this open-source AI initiative has been officially recognized as an "open source foundation."

Breaking down barriers

The initiative’s openness is focused on training LLMs and developing them under irrevocable open licenses, ensuring they remain free from removal clauses and access fees.

To achieve this, OMI plans to establish a governance framework and create specific working groups to promote collaborative development. The Linux Foundation also intends to conduct a survey to gather feedback from the AI community on future research and training models.

Additionally, OMI aims to establish shared standards to enhance interoperability between LLMs and metadata operations. The creation of a transparent, open-source dataset for AI training that all developers can utilize is also on the agenda.

Furthermore, OMI plans to release an alpha test model for targeted red-teaming, with an initial version and scripts for fine-tuning expected before the end of the year.

OPEA initiative also under Linux Foundation

OMI isn’t the first open-source AI initiative backed by the Linux Foundation. Earlier this year, the organization also took the Open Platform for Enterprise AI (OPEA) under its wing. This initiative focuses on developing open standards for Retrieval-Augmented Generation (RAG).

TheClosedsource.AI News

Intel Passed on $1 Billion Investment in OpenAI, Former CEO Doubted Its Value

We've all missed big opportunities in life, and hindsight is always 20/20. However, Intel appears to have made more than its fair share of missteps in terms of "long-term vision" over the past decade, and one of the most financially significant might have been passing on an opportunity with OpenAI.

According to a Reuters report, four sources revealed that between 2017 and 2018, Intel executives engaged in discussions about various ways to collaborate with OpenAI, including the possibility of acquiring a 15% stake in the company for $1 billion.

Two of these sources also mentioned that Intel considered taking an additional 15% stake if it provided hardware to OpenAI at cost.

At the time, sources told Reuters, then-CEO Bob Swan believed that generative AI models wouldn't reach the market anytime soon, making the investment unlikely to yield returns within Intel's desired timeframe.

Now, with OpenAI's market value reportedly around $80 billion, that investment would have been worth about $12 billion today. It also would have placed Intel at the forefront of generative AI development.

OpenAI was reportedly interested in Intel's investment to reduce its reliance on Nvidia's chips and to build its own infrastructure. In hindsight, the missed deal appears to have been a lost opportunity for both companies. While Intel could have invested early in a leading AI company that has thrived during the AI boom, OpenAI continues to seek external funding to design and manufacture its own AI acceleration hardware.

It seems this was a missed opportunity on both sides. Despite OpenAI becoming a major player in AI and a hot topic in the business world, recent reports suggest the company is facing challenges. It's said that OpenAI is currently burning through $700,000 a day to run ChatGPT, with much of this cost tied to Nvidia's AI servers. If Intel had invested in OpenAI, providing the resources needed to develop its own hardware, the situation might have turned out differently.

At least Intel didn't miss out on making chips for the iPhone, right? Oh wait, they did. Yet another opportunity passed by.

Sure, hindsight is always clear, but it seems Intel's management has struggled to make the right decisions over the past decade. Even current CEO Pat Gelsinger has acknowledged these issues, although the company hasn't turned around as hoped under his leadership. Recent troubles for Intel include CPU crashes, mass layoffs, canceled shareholder dividend payments, and production problems, all of which suggest the company is at its lowest point in years.

Once a dominant force in the tech industry, Intel now seems to be in serious trouble. Poor long-term planning, missed opportunities, and questionable decisions have left Intel in a precarious position. While it remains a tech giant, one has to wonder how many more wrong turns it can take before it ends up stranded on the rocks.

Project Strawberry: OpenAI Poised for Major ChatGPT Upgrade

Rumors are swirling that OpenAI is set to release a new model this week, which could mark the debut of the much-anticipated "Project Strawberry"—an AI focused on reasoning.

Following AI discussions on social media lately has felt like a nostalgic trip to strawberry picking, with many accounts sharing images of the fruit, including OpenAI CEO Sam Altman. Altman recently posted a photo of a strawberry plant captioned "I love summer in the garden," which featured four strawberries. This has been interpreted as a hint toward a new version of GPT-4, possibly designed specifically for reasoning, potentially running alongside GPT-4o, which is geared towards creativity and interaction.

The first hints of Project Strawberry emerged in May, suggesting it represents a significant step towards artificial general intelligence (AGI), where AI can think and act like a human. It remains unclear whether Strawberry will be integrated into an updated GPT-4 or introduced as an entirely new model.

While OpenAI has yet to officially release any information on Strawberry, leaked documents indicate it is synonymous with Q*, a training process where an agent makes decisions by estimating the value of actions.

A reasoning-focused AI model could have far-reaching implications, potentially revolutionizing scientific research, improving decision-making in complex fields like healthcare and finance, and driving technological innovation. 

At a user level, such a model could enhance everyday tasks, offering optimized travel plans, personalized health recommendations, and tailored budgets based on spending habits, helping individuals achieve financial freedom.

According to OpenAI, there are five stages on the path to AGI, with the current stage being level one—chatbots capable of natural conversation but lacking advanced problem-solving abilities. Strawberry represents level two—reasoners.

On X, the user iruletheworldmo has been posting strawberry images, hinting that we're on the brink of a 'level two' breakthrough. The user wrote, "welcome to level two. how do you feel? did I make you feel?" to which Altman replied, "amazing tbh," sparking a wave of strawberry-related posts. Altman later tweeted the strawberry plant photo, further fueling the excitement.

Adding to the speculation, Google's Made by Google event is happening this week, and OpenAI has a history of making big announcements during Google events. This has only intensified the anticipation.

While it's uncertain if we'll see a new reasoning model soon, and if so, what form it might take, it feels like we're on the verge of another significant leap in AI capabilities.

In our quest to explore the dynamic and rapidly evolving field of Artificial Intelligence, this newsletter is your go-to source for the latest developments, breakthroughs, and discussions on Generative AI. Each edition brings you the most compelling news and insights from the forefront of Generative AI (GenAI), featuring cutting-edge research, transformative technologies, and the pioneering work of industry leaders.

Highlights from GenAI, OpenAI, and ClosedAI: Dive into the latest projects and innovations from the leading organizations behind some of the most advanced AI models in open-source, closed-sourced AI.

Stay Informed and Engaged: Whether you're a researcher, developer, entrepreneur, or enthusiast, "Towards AGI" aims to keep you informed and inspired. From technical deep-dives to ethical debates, our newsletter addresses the multifaceted aspects of AI development and its implications on society and industry.

Join us on this exciting journey as we navigate the complex landscape of artificial intelligence, moving steadily towards the realization of AGI. Stay tuned for exclusive interviews, expert opinions, and much more!