Top related persons:
Top related locs:
Top related orgs:

Search resuls for: "superalignment"


25 mentions found


Another safety researcher is leaving OpenAI
  + stars: | 2024-10-24 | by ( Darius Rafieyan | ) www.businessinsider.com   time to read: +2 min
Miles Brundage, who advises OpenAI leadership on safety and policy, announced his departure. The AGI Readiness team he oversaw will be disbanded. AdvertisementMiles Brundage, a senior policy advisor and head of the AGI Readiness team at OpenAI, is leaving the company. Brundage is the latest high-profile safety researcher to leave OpenAI. AdvertisementHe's been responsible for some of OpenAI's biggest innovations in safety research, including instituting external red teaming, which involves outside experts looking for potential problems in OpenAI products.
Persons: Miles Brundage, OpenAI, , Brundage, Jan Leike, Ilya Sutskever, Mira Murati, Bob McGrew, Barret Zoph, He's Organizations: AGI, Service Locations: OpenAI
OpenAI is disbanding its "AGI Readiness" team, which advised the company on OpenAI's own capacity to handle increasingly powerful AI and the world's readiness to manage that technology, according to the head of the team. On Wednesday, Miles Brundage, senior advisor for AGI Readiness, announced his departure from the company via a Substack post. Former AGI Readiness team members will be reassigned to other teams, according to the post. Madry will still work on core AI safety work in his new role, OpenAI told CNBC at the time. Leike added that OpenAI must become a "safety-first AGI company."
Persons: OpenAI, Miles Brundage, Brundage, Miles, Mira Murati, Bob McGrew, Barret Zoph —, Aleksander Madry, Madry, reassign Madry, Sam Altman, it's, Altman, Lina Khan, they've, OpenAI's, Ilya Sutskever, Jan Leike, Leike, Greg Brockman, Brockman Organizations: AGI Readiness, CNBC, Security Committee, Google, Microsoft, Meta, Princeton University, Democratic, Federal Trade Commission, Department of Justice, Nvidia Locations: OpenAI
Here's where OpenAI's 11 cofounders are now
  + stars: | 2024-10-03 | by ( Jyoti Mann | ) www.businessinsider.com   time to read: +7 min
Most of OpenAI's cofounders have left the startup, with some going to rivals. Here's where all 11 of its cofounders are now and how many remain at the $157 billion company. Here's where all of the 11 cofounders are now. According to her LinkedIn profile, she used to work at Facebook as an AI research engineer before joining Stripe as a software engineer in 2016. OpenAI and none of its 11 cofounders immediately responded to Business Insider's requests for comment, made outside normal working hours.
Persons: it's, Sam Altman, , OpenAI, Mira Murati, Barret Zoph, Bob McGrew, Ilya Sutskever, John Schulman, Anthropic, Jason Redmond, Altman, John Schulman Schulman, OpenAI's, Jan Leike, Schulman, Wojciech Zaremba Zaremba, Zaremba, Ekon, Musk, Ilya Sutskever Ilya Sutskever, JACK GUEZ, Getty Sutskever, Sustkever, Sustskever, Greg Brockman Brockman, he's, Kingma, Andrej Karpathy Andrej Karpathy, Karpathy, Vicki Cheung Cheung, Elon, Elon Musk, ALAIN JOCARD, Brockman, cofounding OpenAI, Pamela Vagata Vagata, Trevor Blackwell Blackwell, cofounders Organizations: Here's, Service, Getty, OpenAI's Superalignment, OpenAI, Inc, Google DeepMind, Anthropic, Tesla, San Francisco Chronicle, Hearst Newspapers, Getty Images Research, Eureka Labs, SpaceX, The Boring Company, Facebook Locations: AFP, OpenAI, cofounding, Pebblebed
OpenAI exec Kevin Weil told BI the company has a "great bench" after CTO Mira Murati abruptly quit. AdvertisementThe hundreds of attendees at Ray Summit, an AI conference in San Francisco, were originally hoping to hear from Mira Murati, the chief technology officer of OpenAI, on Wednesday. AdvertisementIn a brief interview with Business Insider immediately afterwards, Weil praised Murati while downplaying the impact of her departure. Weil, a former Instagram executive, highlighted Mark Chen as an example of the talent still around at OpenAI. "So I'm really excited to partner with him and I'm really optimistic about what we're building."
Persons: Kevin Weil, Mira Murati, Weil, , Murati, Robert Nishihara, OpenAI, Mira, I've, Mark Chen, Chen, you've, I'm, Jane Street, Sam Altman, Ilya Sutskever, superalignment, Jan Leike, John Schulman, Barret Zoph, Bob McGrew, Bob, we've, We've Organizations: Service, Ray Summit, Business, Integral Technology, MIT Locations: OpenAI, San Francisco, Silicon
A new WSJ report sheds light on the internal tensions at OpenAI amid CTO Mira Murati's exit. The report said execs worried the company could collapse when cofounder Ilya Sutskever left in May. AdvertisementQuestions are swirling about what exactly has been going on at OpenAI amid a string of high-profile executive departures including former CTO Mira Murati. AdvertisementIt also reveals that executives worried about a potential collapse after OpenAI cofounder Ilya Sutskever left in May, and their attempts to woo the AI researcher back before reconsidering the offer. However, the Journal's reporting offers some fresh details about Murati's involvement as CTO amid the crunch to ready and ship new products.
Persons: Mira Murati's, execs, Ilya Sutskever, , Mira Murati, Sam Altman, Murati's, Sutskever, Altman, hadn't, Jan Leike, Greg Brockman, Brockman, Murati, Sarah Friar Organizations: Service, Wall, Business, CNBC
A new WSJ report sheds light on the internal tensions at OpenAI amid CTO Mira Murati's exit. The report said execs worried the company could collapse when cofounder Ilya Sutskever left in May. AdvertisementQuestions are swirling about what exactly has been going on at OpenAI amid a string of high-profile executive departures including former CTO Mira Murati. AdvertisementIt also reveals that executives worried about a potential collapse after OpenAI cofounder Ilya Sutskever left in May, and their attempts to woo the AI researcher back before reconsidering the offer. However, the Journal's reporting offers some fresh details about Murati's involvement as CTO amid the crunch to ready and ship new products.
Persons: Mira Murati's, execs, Ilya Sutskever, , Mira Murati, Sam Altman, Murati's, Sutskever, Altman, hadn't, Jan Leike, Greg Brockman, Brockman, Murati, Sarah Friar Organizations: Service, Wall, Business, CNBC
"We're grateful for John's contributions as a founding team member at OpenAI and his dedicated efforts in advancing alignment research," OpenAI said in a statement to TechCrunch. (I first met John in a cafe in Berkeley in… — Sam Altman (@sama) August 6, 2024This makes Schulman the eighth cofounder who has left OpenAI since its founding in 2015. — Greg Brockman (@gdb) August 6, 2024Besides Schulman, OpenAI has seen multiple exits among its leadership ranks in recent months. In May, OpenAI co-founder and chief scientist Ilya Sutskever announced his resignation from the company. To be sure, not everyone has left OpenAI on such acrimonious terms.
Persons: , John Schulman, Schulman, Altman, OpenAI, John, … — Sam Altman, — Altman, Wojciech Zaremba, Greg Brockman, Brockman, Ilya Sutskever, he'd, OpenAI's, Jan Leike, Leike, Elon Musk, Musk, Roboticist Trevor Blackwell, Vicki Cheung, Durk Kingma, Pamela Vagata, Andrej Karpathy, Karpathy, Tesla, OpenAI didn't Organizations: Service, Business, Superintelligence Inc, OpenAI, Microsoft, Google, ChatGPT, EV giant's, Business Insider Locations: OpenAI, Berkeley,
OpenAI co-founder John Schulman said in a Monday X post that he would leave the Microsoft -backed company and join Anthropic, an artificial intelligence startup with funding from Amazon . In June, OpenAI said Schulman, as head of alignment science, would join a safety and security committee that would provide advice to the board. Schulman has only worked at OpenAI since receiving a Ph.D. in computer science in 2016 from the University of California, Berkeley. The leaders of the superalignment team, Jan Leike and company co-founder Ilya Sutskever, both left this year. Also on Monday, Greg Brockman, another co-founder of OpenAI and its president, announced that he was taking a sabbatical for the rest of the year.
Persons: ChatGPT, OpenAI, John Schulman, Schulman, OpenAI's, Jan Leike, Ilya Sutskever, Leike, Sutskever, Sam Altman, Altman, Tasha McCauley, Helen Toner, Toner, WilmerHale, Greg Brockman Organizations: Darktrace Plc, Microsoft, Amazon, OpenAI, University of California, Anthropic, Superintelligence Inc, Google, Employees, AI Safety Locations: Brooklyn, New York, British, Berkeley, Francisco
OpenAI just lost 3 key leaders, report says
  + stars: | 2024-08-06 | by ( Shubhangi Goel | ) www.businessinsider.com   time to read: +3 min
Read previewThree leaders at OpenAI have just left the company, according to a report by The Information, which cited a person familiar with the matter. Another cofounder, John Schulman, has left OpenAI to join rival AI firm Anthropic. "I've made the difficult decision to leave OpenAI. On X, OpenAI CEO and serial entrepreneur Sam Altman responded to Schulman's departure. Business Insider reported in May that two employees who worked on safety and governance had resigned in recent months.
Persons: , Greg Brockman, Peter Deng, John Schulman, OpenAI, Brockman, Schulman, I've, Sam Altman, Deng, Jan Leike, OpenAI's, Leike, Ilya Sutskever, Daniel Kokotajlo, William Saunders Organizations: Service, Business, TechCrunch, Anthropic Locations: OpenAI, Anthropic
Read previewA former safety employee at OpenAI said the company is following in the footsteps of White Star Line, the company that built the Titanic. Was the path that OpenAI was on more like the Apollo program or more like the Titanic?" Apollo vs TitanicAs Saunders spent more time at OpenAI, he felt leaders were making decisions more akin to "building the Titanic, prioritizing getting out newer, shinier products." OpenAI did not immediately respond to a request for comment sent outside regular business hours by Business Insider. Meanwhile, OpenAI cofounder and former chief scientist Ilya Sutskever, who led the firm's superalignment division, resigned later that month.
Persons: , OpenAI, William Saunders, OpenAI's, Alex Kantrowitz's, Saunders, Ilya Sutskever, JACK GUEZ, Kantrowitz, he's, Saunders — Organizations: Service, White Star Line, Business, Artificial, Intelligence, White Star, Apollo, NASA, BI, OpenAI, Getty, Tech, Apple, Google, Superintelligence Inc Locations: OpenAI, Russia, AFP
This story is available exclusively to Business Insider subscribers. Related storiesOpenAI told Business Insider that the company had "identified and fixed" the "underlying security issue" that led to the breach. AdvertisementStill, the hacking prompted concern inside and outside the company that OpenAI's security is too weak, leaving it open to foreign adversaries like China. He described the company's security as "egregiously insufficient" to protect against theft by foreign actors. A month after OpenAI fired Aschenbrenner, two more of the team's top members quit, and the team effectively dissolved.
Persons: , OpenAI, Leopold Aschenbrenner, Aschenbrenner, Ilya Sustkever, Sam Altman's, Sustkever, Jan Leike, Paul Nakasone Organizations: Service, Business, New York Times, FBI, Employees, NSA, US, Command, Defense Department Locations: China, United States
New York CNN —The OpenAI co-founder who left the high-flying artificial intelligence startup last month has announced his next venture: a company dedicated to building safe, powerful artificial intelligence that could become a rival to his old employer. Ilya Sutskever announced plans for the new company, aptly named Safe Superintelligence Inc., in a post on X Wednesday. Sutskever then worked on Google’s AI research team, before helping to found what would become the maker of ChatGPT. It’s also not clear exactly what the new company thinks of as “safety” in the context of highly powerful artificial intelligence technology. “By safe, we mean safe like nuclear safety as opposed to safe as in ‘trust and safety,’” Sutskever told Bloomberg in an interview published Tuesday.
Persons: Ilya Sutskever, Sutskever, Geoffrey Hinton, Sam Altman, Altman, Kara Swisher, , It’s, ” Sutskever, OpenAI, Jan Leike, Daniel Levy, Daniel Gross Organizations: New, New York CNN, Superintelligence Inc, SSI, Google, CNN, Bloomberg, Apple Locations: New York, OpenAI, , Palo Alto , California, Tel Aviv, Israel
Ilya Sutskever, Russian Israeli-Canadian computer scientist and co-founder and chief scientist of OpenAI, speaks at Tel Aviv University in Tel Aviv, June 5, 2023. OpenAI co-founder Ilya Sutskever, who left the artificial intelligence startup last month, introduced his new AI company, which he's calling Safe Superintelligence, or SSI. "I am starting a new company," Sutskever wrote on X on Wednesday. Altman and Sutskever, along with other directors, clashed over the guardrails OpenAI had put in place in the pursuit of advanced AI. "I deeply regret my participation in the board's actions," Sutskever wrote in a post on X on Nov. 20.
Persons: Ilya Sutskever, OpenAI, Sutskever, Jan Leike, OpenAI's, Leike, Daniel Gross, Daniel Levy, Sam Altman, Altman, we've Organizations: Tel Aviv University, SSI, Microsoft, Apple Locations: Russian Israeli, Canadian, Tel Aviv, Palo Alto , California
Read previewA former OpenAI employee who quit in February spoke out about what led him to quit, and later sign a letter calling for change at AI companies. William Saunders told Business Insider that concerns he raised while working at OpenAI were "not adequately addressed." This story is available exclusively to Business Insider subscribers. Advertisement'Egregiously insufficient'According to Aschenbrenner, OpenAI told employees that he was fired over sharing a document containing safety ideas with external researchers. AdvertisementOpenAI didn't respond to a request for comment from Business Insider.
Persons: , William Saunders, Saunders, they're, Leopold Aschenbrenner, OpenAI's, podcaster Dwarkesh Patel, Aschenbrenner, OpenAI, he'd, Sam Altman, Altman Organizations: Service, Business Locations: OpenAI
Read previewA former OpenAI researcher opened up about how he "ruffled some feathers" by writing and sharing some documents related to safety at the company, and was eventually fired. Leopold Aschenbrenner, who graduated from Columbia University at 19, according to his LinkedIn, worked on OpenAI's superalignment team before he was reportedly "fired for leaking" in April. The AI researcher previously shared the memo with others at OpenAI, "who mostly said it was helpful," he added. Related storiesHR later gave him a warning about the memo, Aschenbrenner said, telling him that it was "racist" and "unconstructive" to worry about China Communist Party espionage. He said he wrote the document a couple of months after the superalignment team was announced, which referenced a four-year planning horizon.
Persons: , Leopold Aschenbrenner, OpenAI's, podcaster Dwarkesh Patel, Aschenbrenner, OpenAI, Sam, Sam Altman Organizations: Service, Columbia University, Business, China Communist Party Locations: OpenAI
It's all unraveling at OpenAI (again)
  + stars: | 2024-06-04 | by ( Madeline Berg | ) www.businessinsider.com   time to read: +10 min
In a statement to Business Insider, an OpenAI spokesperson reiterated the company's commitment to safety, highlighting an "anonymous integrity hotline" for employees to voice their concerns and the company's safety and security committee. Safety second (or third)A common theme of the complaints is that, at OpenAI, safety isn't first — growth and profits are. (In a responding op-ed, current OpenAI board members Bret Taylor and Larry Summers defended Altman and the company's safety standards.) "I have been disagreeing with OpenAI leadership about the company's core priorities for quite some time, until we finally reached a breaking point." (Altman and OpenAI said he recused himself from these deals.)
Persons: , Sam Altman, Daniel Kokotajlo, OpenAI, Altman, Helen Toner, Tasha McCauley, Toner, McCauley, Bret Taylor, Larry Summers, Kokotajlo, Jan Leike, Ilya Sutskever, Leike, Stuart Russell, NDAs, Scarlett Johansson, lawyered, Johansson, " Johansson, I've, Sam Altman — Organizations: Service, New York Times, Business, Times, Twitter, Microsoft, The New York Times, BI, Street, OpenAI, OpenAI's, Apple Locations: OpenAI, Russian, Reddit
Read previewThis has been the week of dueling op-eds from former and current OpenAI board members. Current OpenAI board members Bret Taylor and Larry Summers issued a response to AI safety concerns on Thursday, stating that "the board is taking commensurate steps to ensure safety and security." In the last six months, the two current board members said they had found Altman "highly forthcoming on all relevant issues and consistently collegial with his management team." She also said that the old OpenAI board found out about ChatGPT's release on Twitter. OpenAI dissolved the superalignment safety team before later announcing the formation of a new safety committee.
Persons: , Bret Taylor, Larry Summers, Helen Toner, Tasha McCauley, Sam Altman, Taylor, Summers, Altman, OpenAI, WilmerHale, Toner, Openai, he's, Jan Leike, Ilya Sutskever, Gretchen Krueger, Leike, Krueger Organizations: Service, Business, Twitter, World, Summit
Here's a list of the people, companies, and terms you need to know to talk about AI, in alphabetical order. GPU: A computer chip, short for graphic processing unit, that companies use to train and deploy their AI models. Nvidia's GPUs are used by Microsoft and Meta to run their AI models. Multimodal: The ability for AI models to process text, images, and audio to generate an output. As a profession, prompt engineers are experts in fine tuning AI models on the backend to improve outputs.
Persons: , Sam Altman, Altman, OpenAI's, Dario Amodei, Claude, Demis, Hassabis, Jensen Huang, Satya, Mustafa Suleyman, OpenAI, Elon Musk, Sam Bankman, Peter Thiel, Bard, James Webb, empiricists Organizations: Service, Business, OpenAI, Google, Nvidia, Microsoft, Bing, Meta, James Webb Space Telescope Locations: OpenAI, Anthropic
OpenAI announces new safety board after employee revolt
  + stars: | 2024-05-28 | by ( Brian Fung | ) edition.cnn.com   time to read: +2 min
Washington CNN —OpenAI said Tuesday it has established a new committee to make recommendations to the company’s board about safety and security, weeks after dissolving a team focused on AI safety. In a blog post, OpenAI said the new committee would be led by CEO Sam Altman as well as Bret Taylor, the company’s board chair, and board member Nicole Seligman. The announcement follows the high-profile exit this month of an OpenAI executive focused on safety, Jan Leike. “At the conclusion of the 90 days, the Safety and Security Committee will share their recommendations with the full Board. Following the full Board’s review, OpenAI will publicly share an update on adopted recommendations in a manner that is consistent with safety and security.”
Persons: Washington CNN — OpenAI, OpenAI, Sam Altman, Bret Taylor, Nicole Seligman, Jan Leike, Leike, OpenAI’s, , Ilya Sutskever, Sutskever, Altman’s, Organizations: Washington CNN, CNN, Safety, Security
Jan Leike, one of the lead safety researchers at OpenAI who resigned from the artificial intelligence company earlier this month, said on Tuesday that he's joined rival AI startup Anthropic. Leike announced his resignation from OpenAI on May 15, days before the company dissolved the superalignment group that he co-led. "I'm excited to join @AnthropicAI to continue the superalignment mission," Leike wrote on X. AI safety has gained rapid importance across the tech sector since OpenAI introduced ChatGPT in late 2022, ushering in a boom in generative AI products and investments. The committee will recommend "safety and security decisions for OpenAI projects and operations" to the company's board.
Persons: Jan Leike, he's, Leike, OpenAI, Ilya Sutskever, @AnthropicAI, Sam Altman, Dario Amodei, Daniela Amodei, Claude Organizations: OpenAI, Amazon, Microsoft, Google Locations: OpenAI
Ex-OpenAI exec Jan Leike joined rival AI company Anthropic days after he quit over safety concerns. Leike, who co-led OpenAI's Superalignment team, left less than two weeks ago. AdvertisementOpenAI's former executive Jan Leike announced he's joining its competitor Anthropic. Leike co-led OpenAI's Superalignment team alongside cofounder Ilya Sutskever, who also resigned. The team was tasked with ensuring superintelligence doesn't go rogue and has since been dissolved, with remaining staffers joining the core research team.
Persons: Jan Leike, OpenAI's, OpenAI, , he's, Leike, Ilya Sutskever, superintelligence, @AnthropicAI Organizations: Service, Amazon, Business
A Vox story on Saturday said the company could take back vested equity if departing employees did not sign a non-disparagement agreement. "For a company to threaten to claw back already-vested equity is egregious and unusual," California employment law attorney Chambord Benton-Hayes told Vox. AdvertisementOn Saturday, OpenAI CEO Sam Altman said on X, "Vested equity is vested equity, full stop." We have not and never will take away vested equity, even when people didn't sign the departure documents. Not signing "could impact your equity," OpenAI told one of them, per Vox.
Persons: , Chambord Benton, Hayes, Vox, Sam Altman, Altman, Kelsey Piper's, OpenAI execs, Jason Kwon, Kwon, OpenAI, i've, Scarlett Johansson Organizations: Service, Equity, Business, Vox Locations: California, Vox
OpenAI faces more turmoil as another employee announces she quit over safety concerns. It comes after the resignations of high-profile executives Ilya Sutskever and Jan Leike, who ran its now-dissolved safety research team Superalignment. Krueger wrote, "I resigned a few hours before hearing the news about @ilyasut and @janleike, and I made my decision independently. I resigned a few hours before hearing the news about @ilyasut and @janleike, and I made my decision independently. Kokotajlo said he left after "losing confidence that it [OpenAI] would behave responsibly around the time of AGI."
Persons: Ilya Sutskever, Jan Leike, Gretchen Krueger, Krueger, — Gretchen Krueger, Leike, OpenAI, Daniel Kokotajlo, William Saunders, Kokotajlo, OpenAI didn't Organizations: Business
Read previewThe age of AGI is coming and could be just a few years away, according to OpenAI cofounder John Schulman. Speaking on a podcast with Dwarkesh Patel, Schulman predicted that artificial general intelligence could be achieved in "two or three years." A spokesperson for OpenAI told The Information that the remaining staffers were now part of its core research team. Schulman's comments come amid protest movements calling for a pause on training AI models. Groups such as Pause AI fear that if firms like OpenAI create superintelligent AI models, they could pose existential risks to humanity.
Persons: , John Schulman, Dwarkesh Patel, Schulman, AGI, Elon Musk, OpenAI, Kayla Wood, Jan Leike, Ilya Sutskever Organizations: Service, Business, Tech, Washington Post
AdvertisementIt's a rare admission from Altman, who has worked hard to cultivate an image of being relatively calm amid OpenAI's ongoing chaos. Safety team implosionOpenAI has been in full damage control mode following the exit of key employees working on AI safety. He said the safety team was left "struggling for compute, and it was getting harder and harder to get this crucial research done." Silenced employeesThe implosion of the safety team is a blow for Altman, who has been keen to show he's safety-conscious when it comes to developing super-intelligent AI. The usually reserved Altman even appeared to shade Google, which demoed new AI products the following day.
Persons: , Jan Leike, Ilya Sutskever, Sam Altman, Altman, Leike, Leopold Aschenbrenner, Pavel Izmailov, Daniel Kokotajlo, William Saunders, Cullen O'Keefe, Kokotajlo, Vox, OpenAI, Joe Rogan's, Neel Nanda, i've, Scarlett Johansson, OpenAI didn't Organizations: Service, Business, AGI
Total: 25