October 20, 2021

PC TECH THERAPY

PC Tech Therapy Blog by Daniyal Computer

We read the paper that forced Timnit Gebru out of Google. Here’s what it says

5 min read


Many details of the exact sequence of events that led up to Gebru’s departure are not yet clear; both she and Google have declined to comment beyond their posts on social media. But MIT Technology Review obtained a copy of the research paper from  one of the co-authors, Emily M. Bender, a professor of computational linguistics at the University of Washington. Though Bender asked us not to publish the paper itself because the authors didn’t want such an early draft circulating online, it gives some insight into the questions Gebru and her colleagues were raising about AI that might be causing Google concern.

Titled “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” the paper lays out the risks of large language models—AIs trained on staggering amounts of text data. These have grown increasingly popular—and increasingly large—in the last three years. They are now extraordinarily good, under the right conditions, at producing what looks like convincing, meaningful new text—and sometimes at estimating meaning from language. But, says the introduction to the paper, “we ask whether enough thought has been put into the potential risks associated with developing them and strategies to mitigate these risks.”

The paper

The paper, which builds off the work of other researchers, presents the history of natural-language processing, an overview of four main risks of large language models, and suggestions for further research. Since the conflict with Google seems to be over the risks, we’ve focused on summarizing those here. 

Environmental and financial costs

Training large AI models consumes a lot of computer processing power, and hence a lot of electricity. Gebru and her coauthors refer to a 2019 paper from Emma Strubell and her collaborators on the carbon emissions and financial costs of large language models. It found that their energy consumption and carbon footprint have been exploding since 2017, as models have been fed more and more data.

Strubell’s study found that one language model with a particular type of “neural architecture search” (NAS) method would have produced the equivalent of 626,155 pounds (284 metric tons) of carbon dioxide—about the lifetime output of five average American cars. A version of Google’s language model, BERT, which underpins the company’s search engine, produced 1,438 pounds of CO2 equivalent in Strubell’s estimate—nearly the same as a roundtrip flight between New York City and San Francisco.

Gebru’s draft paper points out that the sheer resources required to build and sustain such large AI models means they tend to benefit wealthy organizations, while climate change hits marginalized communities hardest. “It is past time for researchers to prioritize energy efficiency and cost to reduce negative environmental impact and inequitable access to resources,” they write.

Massive data, inscrutable models

Large language models are also trained on exponentially increasing amounts of text. This means researchers have sought to collect all the data they can from the internet, so there’s a risk that racist, sexist, and otherwise abusive language ends up in the training data.

An AI model taught to view racist language as normal is obviously bad. The researchers, though, point out a couple of more subtle problems. One is that shifts in language play an important role in social change; the MeToo and Black Lives Matter movements, for example, have tried to establish a new anti-sexist and anti-racist vocabulary. An AI model trained on vast swaths of the internet won’t be attuned to the nuances of this vocabulary and won’t produce or interpret language in line with these new cultural norms.

It will also fail to capture the language and the norms of countries and peoples that have less access to the internet and thus a smaller linguistic footprint online. The result is that AI-generated language will be homogenized, reflecting the practices of the richest countries and communities.

Moreover, because the training datasets are so large, it’s hard to audit them to check for these embedded biases. “A methodology that relies on datasets too large to document is therefore inherently risky,” the researchers conclude. “While documentation allows for potential accountability, […] undocumented training data perpetuates harm without recourse.”

Research opportunity costs

The researchers summarize the third challenge as the risk of “misdirected research effort.” Though most AI researchers acknowledge that large language models don’t actually understand language and are merely excellent at manipulating it, Big Tech can make money from models that manipulate language more accurately, so it keeps investing in them. “This research effort brings with it an opportunity cost,” Gebru and her colleagues write. Not as much effort goes into working on AI models that might achieve understanding, or that achieve good results with smaller, more carefully curated datasets (and thus also use less energy).

Illusions of meaning

The final problem with large language models, the researchers say, is that because they’re so good at mimicking real human language, it’s easy to use them to fool people. There have been a few high-profile cases, such as the college student who churned out AI-generated self-help and productivity advice on a blog, which went viral.

The dangers are obvious: AI models could be used to generate misinformation about an election or the covid-19 pandemic, for instance. They can also go wrong inadvertently when used for machine translation. The researchers bring up an example: In 2017, Facebook mistranslated a Palestinian man’s post, which said “good morning” in Arabic, as “attack them” in Hebrew, leading to his arrest.

Why it matters

Gebru and Bender’s paper has six co-authors, four of whom are Google researchers. Bender asked to avoid disclosing their names for fear of repercussions. (Bender, by contrast, is a tenured professor: “I think this is underscoring the value of academic freedom,” she says.)

The paper’s goal, Bender says, was to take stock of the landscape of current research in natural-language processing. “We are working at a scale where the people building the things can’t actually get their arms around the data,” she said. “And because the upsides are so obvious, it’s particularly important to step back and ask ourselves, what are the possible downsides? … How do we get the benefits of this while mitigating the risk?”

In his internal email, Dean, the Google AI head, said one reason the paper “didn’t meet our bar” was that it “ignored too much relevant research.” Specifically, he said it didn’t mention more recent work on how to make large language models more energy-efficient and mitigate problems of bias. 

However, the six collaborators drew on a wide breadth of scholarship. The paper’s citation list, with 128 references, is notably long. “It’s the sort of work that no individual or even pair of authors can pull off,” Bender said. “It really required this collaboration.” 

The version of the paper we saw does also nod to several research efforts on reducing the size and computational costs of large language models, and on measuring the embedded bias of models. It argues, however, that these efforts have not been enough. “I’m very open to seeing what other references we ought to be including,” Bender said.

Nicolas Le Roux, a Google AI researcher in the Montreal office, later noted on Twitter that the reasoning in Dean’s email was unusual. “My submissions were always checked for disclosure of sensitive material, never for the quality of the literature review,” he said.





Source link

Leave a Reply

Your email address will not be published. Required fields are marked *

Missing something

2 min read

The formula for playing Baccarat 24 AI can be used for real. Free Baccarat sa formula.™ Baccarat 24 is available now for free. Baccarat free sa formula, new baccarat betting formula 2021, the most popular formula right now. No need to sit down to write or calculate numbers that are tired of the brain. More accurate than other formulas that have been played at every table Every room is baccarat of Sa and Sexy, and the more bets are placed into every club, it is convenient and fast, showing the results of the second bet per second . The Dragon Baccarat formula is created with a great software system, easy to bet, supports many odds. casino Can be used on both mobile and computer 24 Baccarat betting formulas, what is AI, why do newbies like to use it? free baccarat formula sa The formula for betting on Baccarat 24 according to the table as before, it is Look at the baccarat card layout. dragon card layout ping pong card layout Which is considered a good choice to look at the card layout. But not everyone will remember all the cards. With the changing times, the creation of Baccarat free sa new formula comes up as an artificial intelligence program Browse around this site designed to combine every card deck, every draw statistic, everything is an automatic formula that will analyze the deck immediately. Baccarat 24, Baccarat AI formula, betting formula, Baccarat formula, free sa, extremely accurate, hurry free, just open an account For the Baccarat 24 formula of Youlike555.com It is a form of a program that has been specially developed that can be used in many casinos. Unlike other web AI programs that can only be used by a few casinos No matter which casino you play, you can use our special program. Make your bets accurate and can be stabbed into every club. Free baccarat formula sa created by intelligent program creators, including all statistics, keeping every detail in the live baccarat room. Guarantee that you will get a big money back. Everyone sign up and deposit 100 baht to get the formula for free. Baccarat 24 ways to apply as a member to receive free baccarat formula sa For anyone who is interested in getting the formula for the AI ​​program. To use in betting on Baccarat 24, just register as a member and follow the steps as follows. 1. go to the website link Youlike555.com Then select subscribe. 2. Fill in the name – surname Phone number, bank account number, account name, day, month, year of birth, fill in all fields. The information used in the application must be true. 3. Wait for the approval system Then bring the code and user that the system sends to log in to the system. 4.Deposit money into the betting account with a minimum of 300 and inform the staff. via Line add that you want to receive the AI ​​program Baccarat 24, how to use, free baccarat formula sa 1. Apply as a member Youlike555.com and deposit money 2. Get a link to download the Baccarat 24 program from the staff via LINE AD. and download it to your mobile phone 3. Let the player log in to the casino game and select Baccarat free sa formula, then select the room to play. It is recommended to choose SEXY baccarat. 4. Choose a betting room with a high winning percentage of 100% and a room that matches the formula to be used. 5.Then the Baccarat program formula will be analyzed automatically. Let the players choose to bet according to the formula said. Summary of Baccarat 24 Youlike555.com casino gaming website There is a free baccarat formula sa given away for free. The dragon baccarat formula Formula Baccarat 24 is a formula that can predict accurately display the casino to the second. Make good profits for players. If anyone is looking for a casino game website that has a formula, free baccarat formula, sa give away for free, we recommend. Youlike555.com Casino gambling website with more than 300 games to play, play anywhere, anytime. through online Supports all platforms. There are baccarat dragon formulas available for free, just by signing up. automatic deposit-withdrawal Convenient transactions with all banks, limited free giveaways, hurry up and apply now.

2 min read

The formula for playing Baccarat 24 AI can be used for real. Free Baccarat sa formula.™ Baccarat 24 is available now for free. Baccarat free sa formula, new baccarat betting formula 2021, the most popular formula right now. No need to sit down to write or calculate numbers that are tired of the brain. More accurate than other formulas that have been played at every table Every room is baccarat of Sa and Sexy, and the more bets are placed into every club, it is convenient and fast, showing the results of the second bet per second . The Dragon Baccarat formula is created with a great software system, easy to bet, supports many odds. casino Can be used on both mobile and computer 24 Baccarat betting formulas, what is AI, why do newbies like to use it? free baccarat formula sa The formula for betting on Baccarat 24 according to the table as before, it is Look at the baccarat card layout. dragon card layout ping pong card layout Which is considered a good choice to look at the card layout. But not everyone will remember all the cards. With the changing times, the creation of Baccarat free sa new formula comes up as an artificial intelligence program Browse around this site designed to combine every card deck, every draw statistic, everything is an automatic formula that will analyze the deck immediately. Baccarat 24, Baccarat AI formula, betting formula, Baccarat formula, free sa, extremely accurate, hurry free, just open an account For the Baccarat 24 formula of Youlike555.com It is a form of a program that has been specially developed that can be used in many casinos. Unlike other web AI programs that can only be used by a few casinos No matter which casino you play, you can use our special program. Make your bets accurate and can be stabbed into every club. Free baccarat formula sa created by intelligent program creators, including all statistics, keeping every detail in the live baccarat room. Guarantee that you will get a big money back. Everyone sign up and deposit 100 baht to get the formula for free. Baccarat 24 ways to apply as a member to receive free baccarat formula sa For anyone who is interested in getting the formula for the AI ​​program. To use in betting on Baccarat 24, just register as a member and follow the steps as follows. 1. go to the website link Youlike555.com Then select subscribe. 2. Fill in the name – surname Phone number, bank account number, account name, day, month, year of birth, fill in all fields. The information used in the application must be true. 3. Wait for the approval system Then bring the code and user that the system sends to log in to the system. 4.Deposit money into the betting account with a minimum of 300 and inform the staff. via Line add that you want to receive the AI ​​program Baccarat 24, how to use, free baccarat formula sa 1. Apply as a member Youlike555.com and deposit money 2. Get a link to download the Baccarat 24 program from the staff via LINE AD. and download it to your mobile phone 3. Let the player log in to the casino game and select Baccarat free sa formula, then select the room to play. It is recommended to choose SEXY baccarat. 4. Choose a betting room with a high winning percentage of 100% and a room that matches the formula to be used. 5.Then the Baccarat program formula will be analyzed automatically. Let the players choose to bet according to the formula said. Summary of Baccarat 24 Youlike555.com casino gaming website There is a free baccarat formula sa given away for free. The dragon baccarat formula Formula Baccarat 24 is a formula that can predict accurately display the casino to the second. Make good profits for players. If anyone is looking for a casino game website that has a formula, free baccarat formula, sa give away for free, we recommend. Youlike555.com Casino gambling website with more than 300 games to play, play anywhere, anytime. through online Supports all platforms. There are baccarat dragon formulas available for free, just by signing up. automatic deposit-withdrawal Convenient transactions with all banks, limited free giveaways, hurry up and apply now.

2 min read

The formula for playing Baccarat 24 AI can be used for real. Free Baccarat sa formula.™ Baccarat 24 is available now for free. Baccarat free sa formula, new baccarat betting formula 2021, the most popular formula right now. No need to sit down to write or calculate numbers that are tired of the brain. More accurate than other formulas that have been played at every table Every room is baccarat of Sa and Sexy, and the more bets are placed into every club, it is convenient and fast, showing the results of the second bet per second . The Dragon Baccarat formula is created with a great software system, easy to bet, supports many odds. casino Can be used on both mobile and computer 24 Baccarat betting formulas, what is AI, why do newbies like to use it? free baccarat formula sa The formula for betting on Baccarat 24 according to the table as before, it is Look at the baccarat card layout. dragon card layout ping pong card layout Which is considered a good choice to look at the card layout. But not everyone will remember all the cards. With the changing times, the creation of Baccarat free sa new formula comes up as an artificial intelligence program Browse around this site designed to combine every card deck, every draw statistic, everything is an automatic formula that will analyze the deck immediately. Baccarat 24, Baccarat AI formula, betting formula, Baccarat formula, free sa, extremely accurate, hurry free, just open an account For the Baccarat 24 formula of Youlike555.com It is a form of a program that has been specially developed that can be used in many casinos. Unlike other web AI programs that can only be used by a few casinos No matter which casino you play, you can use our special program. Make your bets accurate and can be stabbed into every club. Free baccarat formula sa created by intelligent program creators, including all statistics, keeping every detail in the live baccarat room. Guarantee that you will get a big money back. Everyone sign up and deposit 100 baht to get the formula for free. Baccarat 24 ways to apply as a member to receive free baccarat formula sa For anyone who is interested in getting the formula for the AI ​​program. To use in betting on Baccarat 24, just register as a member and follow the steps as follows. 1. go to the website link Youlike555.com Then select subscribe. 2. Fill in the name – surname Phone number, bank account number, account name, day, month, year of birth, fill in all fields. The information used in the application must be true. 3. Wait for the approval system Then bring the code and user that the system sends to log in to the system. 4.Deposit money into the betting account with a minimum of 300 and inform the staff. via Line add that you want to receive the AI ​​program Baccarat 24, how to use, free baccarat formula sa 1. Apply as a member Youlike555.com and deposit money 2. Get a link to download the Baccarat 24 program from the staff via LINE AD. and download it to your mobile phone 3. Let the player log in to the casino game and select Baccarat free sa formula, then select the room to play. It is recommended to choose SEXY baccarat. 4. Choose a betting room with a high winning percentage of 100% and a room that matches the formula to be used. 5.Then the Baccarat program formula will be analyzed automatically. Let the players choose to bet according to the formula said. Summary of Baccarat 24 Youlike555.com casino gaming website There is a free baccarat formula sa given away for free. The dragon baccarat formula Formula Baccarat 24 is a formula that can predict accurately display the casino to the second. Make good profits for players. If anyone is looking for a casino game website that has a formula, free baccarat formula, sa give away for free, we recommend. Youlike555.com Casino gambling website with more than 300 games to play, play anywhere, anytime. through online Supports all platforms. There are baccarat dragon formulas available for free, just by signing up. automatic deposit-withdrawal Convenient transactions with all banks, limited free giveaways, hurry up and apply now.

2 min read

The formula for playing Baccarat 24 AI can be used for real. Free Baccarat sa formula.™ Baccarat 24 is available now for free. Baccarat free sa formula, new baccarat betting formula 2021, the most popular formula right now. No need to sit down to write or calculate numbers that are tired of the brain. More accurate than other formulas that have been played at every table Every room is baccarat of Sa and Sexy, and the more bets are placed into every club, it is convenient and fast, showing the results of the second bet per second . The Dragon Baccarat formula is created with a great software system, easy to bet, supports many odds. casino Can be used on both mobile and computer 24 Baccarat betting formulas, what is AI, why do newbies like to use it? free baccarat formula sa The formula for betting on Baccarat 24 according to the table as before, it is Look at the baccarat card layout. dragon card layout ping pong card layout Which is considered a good choice to look at the card layout. But not everyone will remember all the cards. With the changing times, the creation of Baccarat free sa new formula comes up as an artificial intelligence program Browse around this site designed to combine every card deck, every draw statistic, everything is an automatic formula that will analyze the deck immediately. Baccarat 24, Baccarat AI formula, betting formula, Baccarat formula, free sa, extremely accurate, hurry free, just open an account For the Baccarat 24 formula of Youlike555.com It is a form of a program that has been specially developed that can be used in many casinos. Unlike other web AI programs that can only be used by a few casinos No matter which casino you play, you can use our special program. Make your bets accurate and can be stabbed into every club. Free baccarat formula sa created by intelligent program creators, including all statistics, keeping every detail in the live baccarat room. Guarantee that you will get a big money back. Everyone sign up and deposit 100 baht to get the formula for free. Baccarat 24 ways to apply as a member to receive free baccarat formula sa For anyone who is interested in getting the formula for the AI ​​program. To use in betting on Baccarat 24, just register as a member and follow the steps as follows. 1. go to the website link Youlike555.com Then select subscribe. 2. Fill in the name – surname Phone number, bank account number, account name, day, month, year of birth, fill in all fields. The information used in the application must be true. 3. Wait for the approval system Then bring the code and user that the system sends to log in to the system. 4.Deposit money into the betting account with a minimum of 300 and inform the staff. via Line add that you want to receive the AI ​​program Baccarat 24, how to use, free baccarat formula sa 1. Apply as a member Youlike555.com and deposit money 2. Get a link to download the Baccarat 24 program from the staff via LINE AD. and download it to your mobile phone 3. Let the player log in to the casino game and select Baccarat free sa formula, then select the room to play. It is recommended to choose SEXY baccarat. 4. Choose a betting room with a high winning percentage of 100% and a room that matches the formula to be used. 5.Then the Baccarat program formula will be analyzed automatically. Let the players choose to bet according to the formula said. Summary of Baccarat 24 Youlike555.com casino gaming website There is a free baccarat formula sa given away for free. The dragon baccarat formula Formula Baccarat 24 is a formula that can predict accurately display the casino to the second. Make good profits for players. If anyone is looking for a casino game website that has a formula, free baccarat formula, sa give away for free, we recommend. Youlike555.com Casino gambling website with more than 300 games to play, play anywhere, anytime. through online Supports all platforms. There are baccarat dragon formulas available for free, just by signing up. automatic deposit-withdrawal Convenient transactions with all banks, limited free giveaways, hurry up and apply now.