May 5, 2023

Event Report: SWET Talk Shop via Zoom—Chatting about ChatGPT, April 23, 2023

In addition to this summary report of the Talk Shop about ChatGPT, the edited video recording is also available on SWET’s YouTube channel, here.

Note: This event report was produced with the help of two tools from OpenAI, a prominent player in the field of artificial intelligence. Its Whisper program was used to transcribe the Zoom session, and GPT-4 was used to summarize the transcript (special thanks to Tom Gally for adeptly managing these tasks). The GPT-4 summary was then edited by SWET volunteers. The original summary is attached below the edited version.

The SWET Talk Shop held as a Zoom session on April 23, 2023, focusing on the hot topic of ChatGPT, the AI (artificial intelligence) chatbox launched by OpenAI last November 30. It was moderated by Richard Sadowsky, one of SWET’s coordinators in Kansai. The session began with a brief introduction that highlighted ChatGPT’s relevance for SWET members and other wordsmiths. In a real-time poll, 67 percent of the participants said they had used it, and 50 percent that they had used it for work.

The moderator, a freelance translator with an interest in technology, explained that he was not an expert on ChatGPT but had been experimenting with it. He demonstrated the interface, explaining the availability of two models, GPT-3.5 (free) and GPT-4 (paid), and pointed out that the paid version has a higher rating for reasoning and conciseness.
 

Getting Acquainted with ChatGPT

The focus then shifted to the potential applications of ChatGPT in work-related contexts. Participants were asked to give examples based on their own experiences. Various uses were shared, including as a thesaurus, a paraphrasing aid, a subtitling and summarizing tool, a producer of first-draft translations, and a tool for research and linguistic experiments.

The discussion moved on to the concept of “prompt engineering”—the process of tailoring the questions and instructions (“prompts”) that the user feeds into the system in such a way as to optimize the output that it generates. The quality of the prompts has a direct bearing on the quality of the output. Participants emphasized that ChatGPT can make mistakes and may be poor at providing factual information—indeed it sometimes makes up information—but that improvements are expected over time.

One participant asked how to interact with ChatGPT, seeking clarification about how to insert text and ask specific questions. And another suggested that the moderator should demonstrate using ChatGPT by sharing his screen and typing in prompts. The moderator obliged and shared an example in which he had sought synonyms for the term “establishing a charter.” He asked, “What verbs are commonly used to describe the act of a company setting up or establishing an environmental charter?” The chatbot replied with a list of verbs, effectively acting as a thesaurus.

The moderator continued the demonstration by sharing a translation he had been working on. He needed a synonym for the term “societal pressure” in the context of establishing an occupational health, safety, and environmental charter. He input the prompt into ChatGPT and received a list of alternative terms. The demonstration provided a clear example of how ChatGPT can be used to find synonyms and alternative expressions, showcasing its usefulness as a language tool. It also highlighted the ease of interaction with the AI system.

A participant explained the technology behind large language models (LLMs) such as ChatGPT, noting that they are based on deep, complex manipulation of vast amounts of text collected from the Internet. The technology behind them processes the text to generate a static model. When users interact with ChatGPT, they are not engaging with a live, conscious mind but rather with fixed circuits developed from previously processed text. The latest ChatGPT model is based on a massive amount of text, possibly on the order of 20 to 30 trillion words.

Another participant pointed out that while some critics may dismiss ChatGPT as a simple Markov chain, it is much more sophisticated than that, involving deep processing through transformers and attention models. During its development, human checkers were used to interact with the model and flag problematic responses so that the AI system would not produce violent, sexual, or politically controversial content.

Participants touched on the differences between the free and paid versions of ChatGPT, noting that the free version is based on the GPT-3.5 model, whereas the paid version provides access to the more recent GPT-4. It was noted that while GPT-4 may be 30–40 percent better than the free version, the results still depend on the specific task being performed. Some participants pointed out that ChatGPT’s usefulness extends beyond translation; it can also be employed as a tool for search engine optimization (SEO) or in various other ways depending on the user’s imagination and creativity.
 

Potential of ChatGPT as a Wordsmithing Tool

Jason Khoh, translator and managing director of Japanese translation agency, Mochiwa Mochiya Pty Ltd, participating from the wilds of Nagano, remarked that he believes some translators are resistant to using AI tools like ChatGPT out of pride or fear of being replaced by machines. However, he argued that ChatGPT could be an incredible tool for translators, helping them take their work to the next level if they can find the best ways to integrate it into their working processes and use it to cater to their clients’ needs.

The moderator suggested that having a communal store of prompts could be beneficial for users of ChatGPT, as the effectiveness of the AI system often depends on the quality and specificity of the prompts given. He shared an example in which he asked ChatGPT to rewrite a translation in a more persuasive style and was pleased with the resulting improvements in the text.

Participant Tom Gally, who has been closely following ChatGPT since its launch, affirmed that while ChatGPT has potential and can produce good translations, a skilled human translator can still achieve better results. He pointed out that human translators have greater grasp of context and desired tone, and can approach a translation from multiple angles, while AI tools may consider only one possibility.

Tom shared his experience with ChatGPT, highlighting its ability to grasp context better than other translation tools like DeepL and Google Translate. He explained that ChatGPT’s understanding of context can result in more accurate translations for narrative texts, where context is crucial for comprehension. The participants also discussed the importance of using prompts effectively, with one mentioning the possibility of using meta-prompts, with which users can ask ChatGPT to suggest prompts for a specific task.

Tom also touched upon the limitations of ChatGPT in terms of text volume, mentioning that it accepts a maximum of around 4,000 “tokens.” A token may not necessarily be the same as a word, but users might need to break down longer texts into smaller sections to work with the AI tool, and that can result in a loss of cohesion. Tom shared his experience of using the “continue” command when working with longer texts and advised that pasting in the last translated sentence could help ensure better continuity.

The conversation then shifted towards comparing ChatGPT and DeepL. Participants acknowledged that DeepL, particularly its pro version, offers useful features for translators, such as creating custom glossaries and providing alternate translations. However, they also mentioned a French company that is developing a DeepL-like interface with ChatGPT for translation in the background, indicating that the technology continues to advance.

Participant Lynne Riggs (SWET general secretary) emphasized the importance of human judgment and expertise when working with translation tools. Useful as such tools are, professional skills in understanding specific client requirements and context—which AI tools like ChatGPT and DeepL often cannot fully grasp even with skillful prompts—are only going to be more valuable. While it is fine to use these tools to save time and take care of routine or formulaic translation, the unique abilities of human translators will still be needed for performing many translation and writing tasks.

Jason pointed out that AI tools like ChatGPT are still in their infancy and stressed their potential to improve over time. It is important to consider the possibility that jobs like translation may become less secure in the future as AI technology continues to advance. Expert translators and others will have to use these tools and adapt to the way they function. Tom mentioned OpenAI’s development of a plugin system, which could potentially help users automate tasks and improve the AI tool’s capabilities for specific purposes. He also shared his experience with Google’s chatbot, Bard, which currently only supports English and refused to do translation tasks.

During this portion of the discussion, Tom urged that we stay updated about AI tools, noting that being able adapt to them is one way to remain competitive in the translation industry. He emphasized that providing clients with the best possible service and helping them solve language-related problems is crucial for our business. Lynne agreed, stressing that being able to adapt to clients’ specific needs and expectations is part of the added value that translators bring to professional work. Clients need to know whom they can trust and to be able to ask questions and ask for alternatives that will fit their tastes.

The moderator mentioned a recent TED talk about an AI developer’s presentation on a browsing model plugin for ChatGPT that could help with fact-checking. He also reminded us that ChatGPT is not the only AI tool in development; there are other tools like Microsoft Bing, Google Bard, and Open Assistant. The participants also discussed the usefulness of various online resources for providing context and enhancing translations.
 

Is ChatGPT Scary?

Participant Charles DeWolf shared his experience from more than 50 years ago when he was taking a translation course, noting how much translation tools have improved since then. He observed that the translators of earlier decades didn’t feel threatened by the technological advances of their time, such as the development of handy and powerful electronic dictionaries. Why, he asked, do today’s translators find the advent of ChatGPT and its ilk so scary?

Jason suggested that fear depends on one’s perspective. He encouraged us to embrace the technology, explore its potential, and keep up with developments. He mentioned that technology advances rapidly and that there are more ways to make a living today, so it’s essential to remain adaptable. He emphasized his own love for translation but acknowledged the need to consider other options if market forces change the landscape. His main message was to stay open-minded and not be afraid of technological advances.

Tom Gally agreed, emphasizing that people shouldn’t be scared of the technology and should strive to understand and use it to their advantage. But he noted that people’s perspectives could be impacted by how far along they are in their careers, and he admitted that he had initially been frightened by the technology himself. The fear, he said, came from the realization that AI could now do tasks he had once thought only expert humans could do, potentially replacing a significant portion of our work.

Tom said he could understand why some people would be afraid, especially those with repetitive work and a gap between themselves and their clients (such as if they get their work via agencies). He urged us to follow the technology, explore our options, and recognize the value of human interaction in the translation business. Trust between clients and translators remains crucial, and he is convinced that using these new tools can help professionals survive and grow their businesses.
 

Post-editing and Use of AI and MT Tools

Richard then moved the discussion to the topic of post-editing machine translation and how the rates offered for this type of work could be much lower than what people receive for regular translation. Should we be prepared to accept this kind of work and negotiate lower rates for it? Participants expressed differing opinions as they shared their experiences in this connection. Lynne emphasized the importance of educating clients about the quality difference between using machine translation and hiring an experienced professional translator. She thinks that some translators would prefer to do the translation themselves from scratch—even while using AI and MT tools—to maintain the expected quality.

Jason pointed out that large international language service companies might rely on machine translation and post-editing for their large-scale projects, focusing on speed and cost-efficiency. This might not be an attractive market for freelance translators.

The conversation then touched upon leveraging new AI technologies like ChatGPT and finding ways to incorporate them into our work. One participant shared a positive experience where he used machine translation for a technical document at the client’s request and provided a discount, thereby satisfying both sides. However, another participant cautioned against offering discounts too quickly, as post-editing machine translations could potentially result in more work rather than less.
 

Combining Human Craft with LLM Tools

Richard spoke about the importance of communication and trust between translation agencies, clients, and freelance translators. Another participant mentioned the need to establish ethical parameters when using machine translation, as simply copying and pasting text could compromise the quality and trust that clients expect. Translators have to be honest with themselves about the reasons for using machine translation and whether the quality will suffer as a result.

Tom shared his experience transitioning from working as a freelance translator to holding down an in-house job as a translator and faculty member at the University of Tokyo. He highlighted the value of understanding an organization’s policies, decision-making process, and the way it wishes to present itself to the world. This insider knowledge allows a translator to contribute to the organization’s communications in ways that an outside translator or an AI like GPT cannot. He argued that translators could provide significant value by helping clients not only with the language and writing, but also with the message they want to convey. This type of service could be valuable for various organizations, including business corporations, that need help adapting their messages for different audiences.

As the two-hour session drew to a close, participants expressed their appreciation for the insights shared and the supportive community that fosters open communication and learning. There was a call for further dialogue on the SWET-L mailing list, which offers a valuable space for asking questions, sharing resources, and continuing the conversation. The list, at https://groups.io/g/SWET-L, is open to SWET members and non-members alike.

Tom’s offer to use AI tools like OpenAI’s Whisper and GPT-4 to quickly transcribe and summarize the session was met with enthusiasm from others. He and others are enthused by the dramatic advances in AI technology, reminiscing about a time when they had to spend hours manually transcribing audio for their English classes. The conversation then shifted to the topic of transcription services, with some participants discussing the quality of different transcription tools.

In conclusion, participants thanked each other for the fruitful and informative session, and the recording was brought to an end. As people signed off, the Zoom session was left open for those who wanted to spend a bit more time discussing and asking questions.

Originally compiled for the SWET website May 2023 © SWET

*   *  *

Below, please find below the summary compiled using GPT-4, unedited, for comparison. The transcript produced with Whisper was too long for GPT-4 to summarize all at once, so it was divided into about a dozen parts. The initial prompt given to GPT-4 was:

“The following is the first part of the transcript of an online discussion that was held on April 23, 2023. The discussion was organized by a group called SWET, which stands for Society of Writers, Editors, and Translators. Most of the participants are freelancers living in Japan. Summarize this section of the transcript in 3 paragraphs. Do not use any proper names to identify the participants, and refer to any individual as ‘they’ or ‘them.’ ” The prompt for each later part was “The following is the next part of the transcript of that online discussion. Summarize this section of the transcript in 3 paragraphs. Do not use any proper names to identify the participants, and refer to any individual as ‘they’ or ‘them.’ ”

The instruction about names and pronouns was repeated because, in previous summarization tests, GPT-4 had sometimes forgotten instructions given earlier.

*   *   *

The SWET Talk Shop discussion began with a brief introduction, highlighting the relevance of ChatGPT for freelancers and wordsmiths in Japan and abroad. The participants aimed to discuss the potential uses of ChatGPT, learn more about it, and share their experiences with the tool. A poll was conducted to gauge participants' familiarity with ChatGPT, with 67% of the attendees having used it and 50% having used it for work.

The moderator, a freelance translator with an interest in technology, acknowledged that they were not an expert in ChatGPT but had been experimenting with it. They demonstrated the interface, showing the attendees how to choose between different models, such as GPT-3.5 and GPT-4, and mentioned that the paid version provides better reasoning and conciseness. The focus then shifted to the potential applications of ChatGPT in work-related scenarios.

To gather more information on how ChatGPT can be utilized, the moderator suggested asking other participants about their experiences and the various ways in which they have used the tool. The discussion then continued, with participants sharing their perspectives and experiences with ChatGPT.

In this section of the discussion, participants shared various ways they have used ChatGPT. Some found it to be a great thesaurus, helpful for generating ideas to edit and rephrase text, while others used it for paraphrasing, subtitling, and summarizing. One participant mentioned using ChatGPT for translating first drafts from Japanese to English, saving time and producing translations of comparable quality after checking and rewriting. Another participant used it as a research tool to gather different opinions on a theme, while some participants found it helpful for rewriting English texts written by native Japanese speakers or conducting linguistic experiments.

The discussion moved on to the concept of "prompt engineering," which involves designing a series of questions to get better answers from ChatGPT. The quality of the prompts is directly related to the quality of the output. Participants acknowledged that ChatGPT can make mistakes and may be poor at providing factual information, but improvements are expected over time.

One participant asked how to interact with ChatGPT, seeking clarification on whether they should copy a text or ask a specific question. The discussion continued with participants sharing their experiences and providing insights on the different ways they engage with ChatGPT to obtain desired outcomes.

One participant suggested that the moderator should demonstrate using ChatGPT by sharing their screen and typing in prompts. The moderator obliged and shared an example where they sought synonyms for the term "establishing a charter." They asked ChatGPT, "What verbs are commonly used to describe the act of a company setting up or establishing an environmental charter?" The AI provided a list of verbs, effectively acting as a thesaurus.

The moderator continued the demonstration by sharing a translation they had been working on. They needed a synonym for the term "societal pressure" in the context of establishing an occupational health, safety, and environmental charter. They input the prompt into ChatGPT and received a list of alternative terms.

The demonstration provided a clear example of how ChatGPT can be used to find synonyms and alternative expressions, showcasing its usefulness as a language tool. It also highlighted the ease of interaction with the AI through prompts and the ability to switch between different models, such as GPT-3.5 and GPT-4.

The moderator explained how ChatGPT could be used for translation and obtaining different opinions on a topic. One participant wanted to know more about the differences between GPT-3 and GPT-4, and another participant chimed in to explain the technology behind these models.

The participant mentioned that large language models like ChatGPT are based on deep, complex calculations of vast amounts of text collected from the internet. The technology behind them, called transformers, processes the text to generate a static model. When users interact with ChatGPT, they are not engaging with a live conscious mind but rather a fixed circuit developed from previously processed text.

The participant also pointed out that there is an element of randomness involved when interacting with ChatGPT, as the AI provides slightly different answers for the same question each time it is asked. This aspect of the technology can be both useful and challenging, given the rapidly changing nature of information and the need to adapt to new contexts.

The participants shared more about large language models (LLMs), such as ChatGPT, and their capabilities. One participant highlighted that LLMs predict the next word in a sequence based on the context of the input, which has evolved from simple unigrams to complex chains of words. The current ChatGPT model is based on a massive amount of text, potentially on the order of 20 to 30 trillion words.

Another participant pointed out that while some critics may dismiss ChatGPT as a simple Markov chain, it is much more sophisticated than that, involving deep processing through transformers and attention models. During its development, OpenAI used human checkers to interact with the model and flag problematic responses, ultimately making it difficult for the AI to produce violent, sexual, or politically controversial content.

The participants touched on the differences between the free and paid versions of ChatGPT, noting that the free version is based on an earlier model, whereas the paid version provides access to the more recent GPT-4. Due to server capacity limitations, the paid version allows a certain number of responses within a specific time frame.

In this part of the discussion, the participants talked about the differences between the free and paid versions of ChatGPT. They noted that while GPT-4 may be 30-40% better than the free version, it still depends on the specific task being performed. Some participants pointed out that ChatGPT's usefulness extends beyond translation and can be employed as a thesaurus, an SEO tool, or in various other ways depending on the user's imagination and creativity.

One participant shared that they believe some translators are resistant to using AI tools like ChatGPT due to pride or fear of being replaced by machines. However, they argued that ChatGPT could be an incredible tool for translators, helping them take their work to the next level if they can find the best ways to integrate it into their process and cater to their clients' needs.

Another participant suggested that having a communal store of prompts could be beneficial for users of ChatGPT, as the effectiveness of the AI often depends on the quality and specificity of the prompts given. They shared an example where they asked ChatGPT to rewrite a translation in a more persuasive style, and were pleased with the resulting improvements in the text.

The participants agreed that while ChatGPT has potential and can produce good translations, a skilled human translator can still achieve better results. They pointed out that human translators have greater control and capacity, and can approach a translation from multiple angles, while AI tools might offer only one possibility.

One participant shared their experience with ChatGPT, highlighting its ability to grasp context better than other translation tools like DeepL and Google Translate. They explained that ChatGPT's understanding of context can result in more accurate translations for narrative texts, where context is crucial for comprehension. The participants also discussed the importance of using prompts effectively, with one participant mentioning the possibility of using meta-prompts, where users can ask ChatGPT to suggest prompts for a specific task.

They touched upon the limitations of ChatGPT in terms of text volume, mentioning that it accepts a maximum of around 4,000 tokens. A token may not necessarily be the same as a word, so users might need to break down longer texts into smaller sections to work with the AI tool.

During this part of the discussion, the participants elaborated on the limitations of ChatGPT when handling longer texts. One participant explained that they had to break a lengthy text into smaller sections because ChatGPT could not handle it all at once, which resulted in a loss of cohesion. They also shared their experience of using the "continue" command when working with longer texts and advised that pasting the last translated sentence could help ensure better continuity.

The conversation then shifted towards comparing ChatGPT and DeepL. Participants acknowledged that DeepL, particularly its pro version, offers useful features for translators, such as creating custom glossaries and providing alternate translations. However, they also mentioned a French company that is developing a DeepL-like interface with ChatGPT for translation in the background, indicating that the technology continues to advance.

The discussion continued with the participants emphasizing the importance of human judgment and expertise when working with translation tools. They reaffirmed the value of their skills and the need to understand client requirements and context, which AI tools like ChatGPT and DeepL cannot fully grasp without proper prompts. The participants agreed on using these tools to save time on grunt work while also recognizing their unique abilities as human translators.

Participants acknowledged the infancy of AI tools like ChatGPT and the potential for them to improve over time. One participant mentioned that although these tools may not be perfect now, it's important to consider the possibility that jobs like translation may become less secure in the future as AI technology continues to advance. They highlighted the need for experts to use these tools and adapt to the way they function. Another participant mentioned OpenAI's development of a plugin system, which could potentially help users automate tasks and improve the AI's capabilities for specific purposes. They also shared their experience with Google's chatbot, Bard, which currently only supports English and refused to do translation tasks.

During this portion of the discussion, participants agreed on the importance of staying updated with AI tools and adapting to them in order to remain competitive in the industry. They emphasized that providing clients with the best possible service and helping them solve language-related problems is crucial for their business. They also discussed how being able to adapt to clients' specific needs and expectations is part of the added value that translators offer.

One participant mentioned a recent TED talk about an AI developer's presentation on a browsing model plugin for ChatGPT, which could help with fact-checking. They highlighted that ChatGPT is not the only AI tool in development, pointing out that there are other tools like Microsoft Bing, Google Bard, and Open Assistant. The participants also discussed the usefulness of various online resources for providing context and enhancing translations.

A participant shared their experience from more than 50 years ago when they were taking a translation course, mentioning how much translation tools have improved since then. They expressed that translators didn't feel threatened by previous advancements, but the new technology seems to be causing fear. The participant asked what makes this new technology so scary.

Another participant responded by suggesting that fear depends on perspective. They encouraged embracing the technology, exploring its potential, and keeping up with developments. The participant mentioned that technology advances rapidly and that there are more ways to make a living today, so it's essential to remain adaptable. They emphasized their love for translation but acknowledged the need to consider other options if market forces change the landscape. The main takeaway was to stay open-minded and not be scared of technological advancements.

A participant emphasized that people shouldn't be scared of the technology and should strive to understand and use it to their advantage. They agreed that the point in one's career could impact their perspective, admitting they had been initially frightened by the technology. They explained that their fear came from the realization that AI could now do tasks they once thought only expert humans could do, potentially replacing a significant portion of their work.

Another participant shared that they could understand why some people would be afraid, especially those with repetitive work and a gap between them and their clients. They argued that it's essential to follow the technology, explore possibilities, and recognize the value of human interaction in the translation business. Trust between clients and translators remains crucial, and using new tools can help professionals survive and grow their businesses.

The discussion touched upon the topic of post-editing machine translation and how the offered rates for this type of work could be much lower. The willingness to accept this kind of work and negotiate rates was mentioned, and participants seemed to have different opinions on the matter.

Participants shared their experiences with machine translation and post-editing. One participant emphasized the importance of educating clients about the quality difference between using machine translation and hiring a professional translator. They argued that a translator would prefer to do the translation from scratch to maintain the expected quality.

Another participant pointed out that large international language service companies might rely on machine translation and post-editing for their large-scale projects, focusing on speed and cost-efficiency. However, for individual freelancers, this might not be an attractive market. One participant mentioned that post-editing machine translations could be a tedious task, especially with older translation tools.

The conversation then touched upon leveraging new AI technologies like ChatGPT and finding ways to incorporate them into their work. One participant shared a positive experience where they used machine translation for a technical document at the client's request and provided a discount. However, another participant cautioned against offering discounts too quickly, as working with machine translations could potentially result in more work rather than less.

Participants focused on the importance of communication and trust between translation agencies, clients, and freelance translators. One participant mentioned the need to establish ethical parameters when using machine translation, as simply copying and pasting text could compromise the quality and trust that clients expect. They emphasized that translators should be honest with themselves about the reasons for using machine translation and whether the quality would suffer as a result.

Another participant shared their experience transitioning from a freelance translator to an in-house translator at a university. They highlighted the value of understanding an organization's policies, decision-making process, and desired presentation to the world. This insider knowledge allows translators to contribute to the organization's communications in ways that an outside translator or an AI like GPT might struggle with. The participant argued that translators could provide significant value by helping clients not only with the language and writing, but also with the message they want to convey.

One participant suggested that freelance translators could benefit from building relationships with clients in which they serve as consultants, not only for how clients express their messages but also for what they say. This type of service could be valuable for various organizations, including private companies, that need help adapting their messages for different audiences.

Towards the end of the discussion, participants expressed their appreciation for the insights shared and the supportive community that fosters open communication and learning. They encouraged further dialogue on the mailing list, which offers a valuable space for asking questions, sharing resources, and continuing the conversation. One participant shared a link to the mailing list in the chat, and another reminded everyone that they can download the chat for future reference.

An interesting anecdote emerged when a participant offered to use AI tools like OpenAI's Whisper and GPT-4 to quickly transcribe and summarize the session, which was met with enthusiasm from others. The participant highlighted the impressive advancements in AI technology, reminiscing about a time when they had to spend hours manually transcribing audio for their English classes. The conversation then shifted to the topic of transcription services, with some participants discussing the quality of different transcription tools.

In conclusion, participants thanked each other for the fruitful and informative session, and the recording was brought to an end. However, the meeting was left open for those who wanted to continue discussing and asking questions.