What is ChatGPT?<\/strong><\/h2>\n\n\n\nLet\u2019s ask ChatGPT itself what it is and what sources it uses to give responses:<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Fig.-1-1-1.png\")
<\/a>Fig. 1 What is ChatGPT and what sources it uses \u2013 AI response<\/figcaption><\/figure>\n\n\n\nApplying AI in testing<\/strong><\/h2>\n\n\n\nTest cases<\/strong><\/h3>\n\n\n\nLet\u2019s begin with the issue, which most of the testers encounter at the beginning of their career path. Let\u2019s see how ChatGPT deals with two questions regarding the creation of test cases. I asked ChatGPT about logging to a webpage and a mobile app test cases. You can see the responses below.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.2-1024x702.png\")
<\/a>Fig. 2 ChatGPT response to the question regarding creating a test case for logging to a webpage<\/figcaption><\/figure>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.3-1014x1024.png\")
<\/a>Fig. 3 ChatGPT response to the question regarding creating a test case for mobile ap<\/figcaption><\/figure>\n\n\n\nCurrently I am working on a mobile app project in a medical market. That is why I have added more details to the query above.<\/p>\n\n\n\n
Here is the result:<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/4-1.png\")
<\/a>Fig. 4 ChatGPT response to the question regarding creating a test case for medical services mobile app<\/figcaption><\/figure>\n\n\n\nChatGPT\u2019s response was surprising to me, because – apart from a couple differences \u2013 I could use this test case in the project.<\/p>\n\n\n\n
Questions on automated tests<\/strong><\/h3>\n\n\n\nWe will now look at how AI will deal with a harder subject regarding automated tests.<\/p>\n\n\n\n
Let\u2019s begin with something easier, so a query on how to open a webpage using Selenium.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.5-1024x569.png\")
<\/a>Fig. 5 ChatGPT response to the question regarding opening a webpage in Selenium<\/figcaption><\/figure>\n\n\n\nThe code above can be considered correct for the Python language.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/6-1.png\")
<\/a>Fig. 6 Example of Selenium documentation<\/figcaption><\/figure>\n\n\n\nWe did not specify the name of any programming language in our query. Let\u2019s see what happens when we specify it and indicate a specific language.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.7-1024x930.png\")
<\/a>Fig. 7 ChatGPT response to the question regarding opening a webpage in Selenium in Java<\/figcaption><\/figure>\n\n\n\nI have checked if the above code works. Below is the result:<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.8-1024x629.png\")
<\/a>Fig. 8 Example of running the code generated by ChatGPT<\/figcaption><\/figure>\n\n\n\nI did not have any problem with running the above code, and the outcome was as expected, which is Sii website<\/a> has been properly opened in the browser. <\/p>\n\n\n\nOf course, we can continue to improve our query. Below I have modified it, so that ChatGPT tries to verify if the webpage title is correct.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.9-873x1024.png\")
<\/a>Fig. 9 ChatGPT response to the question regarding opening a webpage in Selenium and verifying the webpage title<\/figcaption><\/figure>\n\n\n\nNaturally, in real life we should not use such code \u2013 instead use an assertion<\/strong>. Therefore, now will ask ChatGPT for an assertion.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.10-904x1024.png\")
<\/a>Fig. 10 ChatGPT response to the question regarding opening a webpage in Selenium and verifying the webpage title with the use of an assertion<\/figcaption><\/figure>\n\n\n\nIn the examples above ChatGPT has no problem with indicating libraries, which should be imported for the code to work properly. Still, I was surprised with the fact, that the tool has also managed to prepare the code with the inclusion of assertion in TestNG.<\/p>\n\n\n\n
Question on Robot Framework<\/strong><\/h3>\n\n\n\nUnfortunately, ChatGPT not always generates response in such a way. In case of a query regarding Robot Framework, the tool did not indicate the necessary library to import.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/11-1.png\")
<\/a>Fig. 11 ChatGPT response to the question regarding opening a webpage in Robot Framework<\/figcaption><\/figure>\n\n\n\nIn this case the SeleniumLibrary should be imported.<\/p>\n\n\n\n
Questions on Selenium, Cypress and Playwright<\/strong><\/strong><\/h3>\n\n\n\nIn case of Selenium the most frequently asked question concerns the differences between implicit wait and explicit wait. Therefore, it is worth checking if artificial intelligence will handle that issue.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.12-1024x992.png\")
<\/a>Fig. 12 ChatGPT response to the question regarding the difference between implicit wait and explicit wait<\/figcaption><\/figure>\n\n\n\nI leave the evaluation of this response to you An excerpt from the documentation on this issue<\/a> may help you with it.<\/p>\n\n\n\nChatGPT is able to respond to even more complicated queries:<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.13-536x1024.png\")
<\/a>Fig. 13 ChatGPT response to the question regarding opening a webpage in Selenium with the use f PageObject Pattern<\/figcaption><\/figure>\n\n\n\nA tester does not live by Selenium alone Nowadays, Cypress is also frequently used and Playwright framework is gaining popularity. Can ChatGPT be helpful in this case?<\/p>\n\n\n\n
In case of Cypress the code generated by the tool was working and performed the test correctly:<\/p>\n\n\n
\ndescribe('www.sii.pl', () => {\n it('should open the homepage', () => {\n cy.visit('https:\/\/www.sii.pl');\n \/\/ Assert that the page has loaded successfully\n cy.title().should('eq', 'Strona g\u0142\u00f3wna - SII');\n });\n});\n<\/pre><\/div>\n\n\nExample:<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.14-1024x578.png\")
<\/a>Fig. 14 Example of running the code, which ChatGPT generated for Cypress<\/figcaption><\/figure>\n\n\n\nUnfortunately, the situation was not that colorful in case of Playwright. ChatGPT prepared the following code for me:<\/p>\n\n\n
\nconst { chromium } = require('playwright');\ndescribe('www.sii.pl', () => {\n it('should open the homepage', async () => {\n const browser = await chromium.launch();\n const page = await browser.newPage();\n await page.goto('https:\/\/www.sii.pl');\n \/\/ Assert that the page has loaded successfully\n expect(await page.title()).toEqual('Strona g\u0142\u00f3wna - SII');\n await browser. Close();\n });\n});\n<\/pre><\/div>\n\n\nTaking into consideration the format, which ChatGPT has adopted, in my opinion the code should look like this:<\/p>\n\n\n
\nimport { test, expect } from '@playwright\/test'\ntest.describe.only('www.sii.pl', () => {\n test('should open the homepage', async ({ page }) => {\n await page.goto('https:\/\/www.sii.pl');\n \/\/ Assert that the page has loaded successfully\n expect(await page.title()).toEqual('Strona g\u0142\u00f3wna - SII');\n })\n})\n<\/pre><\/div>\n\n\nOf course, in my research I have also checked other issues concerning automated tests and programming, including queries regarding:<\/p>\n\n\n\n
\n- Appium,<\/li>\n\n\n\n
- HTML,<\/li>\n\n\n\n
- JavaScript.<\/li>\n<\/ul>\n\n\n\n
ChatGPT and the issue of accessibility<\/strong><\/h2>\n\n\n\nFor some time now I have been very interested in the topic of the accessibility. The WCAG standards are connected with that topic.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.15-749x1024-1.png\")
Fig 15. ChatGPT response to the question regarding accessibility in accordance with WCAG standards<\/figcaption><\/figure>\n\n\n\nChatGPT responds in different languages. For me, the response in Polish was surprising. I also think, that creating web content accessible also for infants is very hard <\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.16-1024x904.png\")
<\/a>Fig. 16 ChatGPT response which recommends creating the content in accordance with WCAG standard, accessible i.e. for infants (PL)<\/figcaption><\/figure>\n\n\n\nOf course, also in the case of A11Y (accessibility) we can ask more specific questions.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.17-1024x698.png\")
<\/a>Fig. 17 ChatGPT response to the question regarding adding an aria label in HTML<\/figcaption><\/figure>\n\n\n\nIt should be mentioned here that ChatGPT does not always work as expected. In the example above, I asked the question in English and the answer was given in Polish.<\/p>\n\n\n\n
Controversies concerning AI<\/strong><\/strong><\/h2>\n\n\n\nThe main goal of the article was to present AI possibilities in the context of testing, however I think that the controversies concerning artificial intelligence are worth mentioning.<\/p>\n\n\n\n
Copyrights<\/strong><\/strong><\/h3>\n\n\n\nMedia and authors report, that AI very often uses other people\u2019s work<\/a>. Midjourney creator said to the Forbes magazine:<\/a><\/p>\n\n\n\n\nThere isn\u2019t really a way to get a hundred million images and know where they\u2019re coming from. It would be cool if images had metadata embedded in them about the copyright owner or something. But that’s not a thing; there’s not a registry. There\u2019s no way to find a picture on the Internet, and then automatically trace it to an owner and then have any way of doing anything to authenticate it.<\/em><\/p>\n<\/blockquote>\n\n\n\nFrom these words, in my opinion, confirmation of the use of other people\u2019s works by artificial intelligence can be deducted. It has a particular significance in the context of the yet another case of right infringement of millions of creators<\/a>, who decided to file a class action lawsuit.<\/p>\n\n\n\nEducation and critical thinking<\/strong><\/strong><\/h3>\n\n\n\nSchools in New York have restricted access to ChatGPT<\/a>. Below are the words of Jenny Lyle, spokesperson for the U.S. Department of Education:<\/p>\n\n\n\n\nDue to concerns about negative impacts on student learning, and concerns regarding the safety and accuracy of content, access to ChatGPT is restricted on New York City Public Schools\u2019 networks and devices.\u201d<\/em><\/p>\n<\/blockquote>\n\n\n\n\nWhile the tool may be able to provide quick and easy answers to questions, it does not build critical-thinking and problem-solving skills, which are essential for academic and lifelong success<\/em>.<\/p>\n<\/blockquote>\n\n\n\nI do agree with these words. I live in a city without access to any sea or lake, and ChatGPT tries to convince me, that there are beautiful beaches in Reda.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/Ryc.18-1024x828.jpeg\")
<\/a>Fig. 18 ChatGPT response to the question about Reda town<\/figcaption><\/figure>\n\n\n\nCan AI be relied upon?<\/strong><\/h2>\n\n\n\nQuestions arise:<\/p>\n\n\n\n
\n- Can AI replace a human being?<\/li>\n\n\n\n
- Can responses generated by these tools be trusted?<\/li>\n<\/ul>\n\n\n\n
Programs using artificial intelligence still often give wrong answers or provide outdated information.<\/p>\n\n\n\n
Above, I have presented and example of the code, that needed to be corrected in order to work, as well as a response, in which ChatGPT proposed ensuring website accessibility to infants.<\/p>\n\n\n\n
In my history of using the tool I have also probably encountered a situation of providing outdated information. I asked about differences between Cypress and Playwright. The response was as follows:<\/p>\n\n\n\n
\nPlaywright supports a wider range of browsers than Cypress, including Chromium (used by Google Chrome), Firefox, and WebKit (used by Safari). Cypress currently only supports Chromium.<\/p>\n<\/blockquote>\n\n\n\n
I decided to look into that response and verify it. On the official website, Cypress informs about the support of many browsers and engines, i.e.: Chrome, Firefox, Edge, Webkit. In this case, we may be dealing with a ChatGPT limitation. ChatGPT contains information until 2021.<\/p>\n\n\n\n![\"\"](\"https:\/\/sii.ua\/blog\/wp-content\/uploads\/2024\/01\/20-1.png\")
<\/a>Fig. 19 ChatGPT limitations<\/figcaption><\/figure>\n\n\n\nTherefore, it is a confirmation, that a human and their control are essential.<\/p>\n\n\n\n
Other IT areas are also considering the possibilities of using AI in their work. On YouTube you can find materials on the use of artificial intelligence in the process of creating a website<\/a> or application<\/a>. It is worth looking into what they say.<\/p>\n\n\n\n
<\/a>Applying AI in testing<\/strong><\/h2>\n\n\n\nTest cases<\/strong><\/h3>\n\n\n\nLet\u2019s begin with the issue, which most of the testers encounter at the beginning of their career path. Let\u2019s see how ChatGPT deals with two questions regarding the creation of test cases. I asked ChatGPT about logging to a webpage and a mobile app test cases. You can see the responses below.<\/p>\n\n\n\n<\/a>Fig. 2 ChatGPT response to the question regarding creating a test case for logging to a webpage<\/figcaption><\/figure>\n\n\n\n<\/a>Fig. 3 ChatGPT response to the question regarding creating a test case for mobile ap<\/figcaption><\/figure>\n\n\n\nCurrently I am working on a mobile app project in a medical market. That is why I have added more details to the query above.<\/p>\n\n\n\n
Here is the result:<\/p>\n\n\n\n<\/a>Fig. 4 ChatGPT response to the question regarding creating a test case for medical services mobile app<\/figcaption><\/figure>\n\n\n\nChatGPT\u2019s response was surprising to me, because – apart from a couple differences \u2013 I could use this test case in the project.<\/p>\n\n\n\n
Questions on automated tests<\/strong><\/h3>\n\n\n\nWe will now look at how AI will deal with a harder subject regarding automated tests.<\/p>\n\n\n\n
Let\u2019s begin with something easier, so a query on how to open a webpage using Selenium.<\/p>\n\n\n\n<\/a>Fig. 5 ChatGPT response to the question regarding opening a webpage in Selenium<\/figcaption><\/figure>\n\n\n\nThe code above can be considered correct for the Python language.<\/p>\n\n\n\n<\/a>Fig. 6 Example of Selenium documentation<\/figcaption><\/figure>\n\n\n\nWe did not specify the name of any programming language in our query. Let\u2019s see what happens when we specify it and indicate a specific language.<\/p>\n\n\n\n<\/a>Fig. 7 ChatGPT response to the question regarding opening a webpage in Selenium in Java<\/figcaption><\/figure>\n\n\n\nI have checked if the above code works. Below is the result:<\/p>\n\n\n\n<\/a>Fig. 8 Example of running the code generated by ChatGPT<\/figcaption><\/figure>\n\n\n\nI did not have any problem with running the above code, and the outcome was as expected, which is Sii website<\/a> has been properly opened in the browser. <\/p>\n\n\n\nOf course, we can continue to improve our query. Below I have modified it, so that ChatGPT tries to verify if the webpage title is correct.<\/p>\n\n\n\n<\/a>Fig. 9 ChatGPT response to the question regarding opening a webpage in Selenium and verifying the webpage title<\/figcaption><\/figure>\n\n\n\nNaturally, in real life we should not use such code \u2013 instead use an assertion<\/strong>. Therefore, now will ask ChatGPT for an assertion.<\/p>\n\n\n\n<\/a>Fig. 10 ChatGPT response to the question regarding opening a webpage in Selenium and verifying the webpage title with the use of an assertion<\/figcaption><\/figure>\n\n\n\nIn the examples above ChatGPT has no problem with indicating libraries, which should be imported for the code to work properly. Still, I was surprised with the fact, that the tool has also managed to prepare the code with the inclusion of assertion in TestNG.<\/p>\n\n\n\n
Question on Robot Framework<\/strong><\/h3>\n\n\n\nUnfortunately, ChatGPT not always generates response in such a way. In case of a query regarding Robot Framework, the tool did not indicate the necessary library to import.<\/p>\n\n\n\n<\/a>Fig. 11 ChatGPT response to the question regarding opening a webpage in Robot Framework<\/figcaption><\/figure>\n\n\n\nIn this case the SeleniumLibrary should be imported.<\/p>\n\n\n\n
Questions on Selenium, Cypress and Playwright<\/strong><\/strong><\/h3>\n\n\n\nIn case of Selenium the most frequently asked question concerns the differences between implicit wait and explicit wait. Therefore, it is worth checking if artificial intelligence will handle that issue.<\/p>\n\n\n\n<\/a>Fig. 12 ChatGPT response to the question regarding the difference between implicit wait and explicit wait<\/figcaption><\/figure>\n\n\n\nI leave the evaluation of this response to you An excerpt from the documentation on this issue<\/a> may help you with it.<\/p>\n\n\n\nChatGPT is able to respond to even more complicated queries:<\/p>\n\n\n\n<\/a>Fig. 13 ChatGPT response to the question regarding opening a webpage in Selenium with the use f PageObject Pattern<\/figcaption><\/figure>\n\n\n\nA tester does not live by Selenium alone Nowadays, Cypress is also frequently used and Playwright framework is gaining popularity. Can ChatGPT be helpful in this case?<\/p>\n\n\n\n
In case of Cypress the code generated by the tool was working and performed the test correctly:<\/p>\n\n\n
\ndescribe('www.sii.pl', () => {\n it('should open the homepage', () => {\n cy.visit('https:\/\/www.sii.pl');\n \/\/ Assert that the page has loaded successfully\n cy.title().should('eq', 'Strona g\u0142\u00f3wna - SII');\n });\n});\n<\/pre><\/div>\n\n\nExample:<\/p>\n\n\n\n<\/a>Fig. 14 Example of running the code, which ChatGPT generated for Cypress<\/figcaption><\/figure>\n\n\n\nUnfortunately, the situation was not that colorful in case of Playwright. ChatGPT prepared the following code for me:<\/p>\n\n\n
\nconst { chromium } = require('playwright');\ndescribe('www.sii.pl', () => {\n it('should open the homepage', async () => {\n const browser = await chromium.launch();\n const page = await browser.newPage();\n await page.goto('https:\/\/www.sii.pl');\n \/\/ Assert that the page has loaded successfully\n expect(await page.title()).toEqual('Strona g\u0142\u00f3wna - SII');\n await browser. Close();\n });\n});\n<\/pre><\/div>\n\n\nTaking into consideration the format, which ChatGPT has adopted, in my opinion the code should look like this:<\/p>\n\n\n
\nimport { test, expect } from '@playwright\/test'\ntest.describe.only('www.sii.pl', () => {\n test('should open the homepage', async ({ page }) => {\n await page.goto('https:\/\/www.sii.pl');\n \/\/ Assert that the page has loaded successfully\n expect(await page.title()).toEqual('Strona g\u0142\u00f3wna - SII');\n })\n})\n<\/pre><\/div>\n\n\nOf course, in my research I have also checked other issues concerning automated tests and programming, including queries regarding:<\/p>\n\n\n\n
\n- Appium,<\/li>\n\n\n\n
- HTML,<\/li>\n\n\n\n
- JavaScript.<\/li>\n<\/ul>\n\n\n\n
ChatGPT and the issue of accessibility<\/strong><\/h2>\n\n\n\nFor some time now I have been very interested in the topic of the accessibility. The WCAG standards are connected with that topic.<\/p>\n\n\n\nFig 15. ChatGPT response to the question regarding accessibility in accordance with WCAG standards<\/figcaption><\/figure>\n\n\n\nChatGPT responds in different languages. For me, the response in Polish was surprising. I also think, that creating web content accessible also for infants is very hard <\/p>\n\n\n\n<\/a>Fig. 16 ChatGPT response which recommends creating the content in accordance with WCAG standard, accessible i.e. for infants (PL)<\/figcaption><\/figure>\n\n\n\nOf course, also in the case of A11Y (accessibility) we can ask more specific questions.<\/p>\n\n\n\n<\/a>Fig. 17 ChatGPT response to the question regarding adding an aria label in HTML<\/figcaption><\/figure>\n\n\n\nIt should be mentioned here that ChatGPT does not always work as expected. In the example above, I asked the question in English and the answer was given in Polish.<\/p>\n\n\n\n
Controversies concerning AI<\/strong><\/strong><\/h2>\n\n\n\nThe main goal of the article was to present AI possibilities in the context of testing, however I think that the controversies concerning artificial intelligence are worth mentioning.<\/p>\n\n\n\n
Copyrights<\/strong><\/strong><\/h3>\n\n\n\nMedia and authors report, that AI very often uses other people\u2019s work<\/a>. Midjourney creator said to the Forbes magazine:<\/a><\/p>\n\n\n\n\nThere isn\u2019t really a way to get a hundred million images and know where they\u2019re coming from. It would be cool if images had metadata embedded in them about the copyright owner or something. But that’s not a thing; there’s not a registry. There\u2019s no way to find a picture on the Internet, and then automatically trace it to an owner and then have any way of doing anything to authenticate it.<\/em><\/p>\n<\/blockquote>\n\n\n\nFrom these words, in my opinion, confirmation of the use of other people\u2019s works by artificial intelligence can be deducted. It has a particular significance in the context of the yet another case of right infringement of millions of creators<\/a>, who decided to file a class action lawsuit.<\/p>\n\n\n\nEducation and critical thinking<\/strong><\/strong><\/h3>\n\n\n\nSchools in New York have restricted access to ChatGPT<\/a>. Below are the words of Jenny Lyle, spokesperson for the U.S. Department of Education:<\/p>\n\n\n\n\nDue to concerns about negative impacts on student learning, and concerns regarding the safety and accuracy of content, access to ChatGPT is restricted on New York City Public Schools\u2019 networks and devices.\u201d<\/em><\/p>\n<\/blockquote>\n\n\n\n\nWhile the tool may be able to provide quick and easy answers to questions, it does not build critical-thinking and problem-solving skills, which are essential for academic and lifelong success<\/em>.<\/p>\n<\/blockquote>\n\n\n\nI do agree with these words. I live in a city without access to any sea or lake, and ChatGPT tries to convince me, that there are beautiful beaches in Reda.<\/p>\n\n\n\n<\/a>Fig. 18 ChatGPT response to the question about Reda town<\/figcaption><\/figure>\n\n\n\nCan AI be relied upon?<\/strong><\/h2>\n\n\n\nQuestions arise:<\/p>\n\n\n\n
\n- Can AI replace a human being?<\/li>\n\n\n\n
- Can responses generated by these tools be trusted?<\/li>\n<\/ul>\n\n\n\n
Programs using artificial intelligence still often give wrong answers or provide outdated information.<\/p>\n\n\n\n
Above, I have presented and example of the code, that needed to be corrected in order to work, as well as a response, in which ChatGPT proposed ensuring website accessibility to infants.<\/p>\n\n\n\n
In my history of using the tool I have also probably encountered a situation of providing outdated information. I asked about differences between Cypress and Playwright. The response was as follows:<\/p>\n\n\n\n
\nPlaywright supports a wider range of browsers than Cypress, including Chromium (used by Google Chrome), Firefox, and WebKit (used by Safari). Cypress currently only supports Chromium.<\/p>\n<\/blockquote>\n\n\n\n
I decided to look into that response and verify it. On the official website, Cypress informs about the support of many browsers and engines, i.e.: Chrome, Firefox, Edge, Webkit. In this case, we may be dealing with a ChatGPT limitation. ChatGPT contains information until 2021.<\/p>\n\n\n\n<\/a>Fig. 19 ChatGPT limitations<\/figcaption><\/figure>\n\n\n\nTherefore, it is a confirmation, that a human and their control are essential.<\/p>\n\n\n\n
Other IT areas are also considering the possibilities of using AI in their work. On YouTube you can find materials on the use of artificial intelligence in the process of creating a website<\/a> or application<\/a>. It is worth looking into what they say.<\/p>\n\n\n\n
Let\u2019s begin with the issue, which most of the testers encounter at the beginning of their career path. Let\u2019s see how ChatGPT deals with two questions regarding the creation of test cases. I asked ChatGPT about logging to a webpage and a mobile app test cases. You can see the responses below.<\/p>\n\n\n\n Currently I am working on a mobile app project in a medical market. That is why I have added more details to the query above.<\/p>\n\n\n\n Here is the result:<\/p>\n\n\n\n ChatGPT\u2019s response was surprising to me, because – apart from a couple differences \u2013 I could use this test case in the project.<\/p>\n\n\n\n We will now look at how AI will deal with a harder subject regarding automated tests.<\/p>\n\n\n\n Let\u2019s begin with something easier, so a query on how to open a webpage using Selenium.<\/p>\n\n\n\n The code above can be considered correct for the Python language.<\/p>\n\n\n\n We did not specify the name of any programming language in our query. Let\u2019s see what happens when we specify it and indicate a specific language.<\/p>\n\n\n\n I have checked if the above code works. Below is the result:<\/p>\n\n\n\n I did not have any problem with running the above code, and the outcome was as expected, which is Sii website<\/a> has been properly opened in the browser. <\/p>\n\n\n\n Of course, we can continue to improve our query. Below I have modified it, so that ChatGPT tries to verify if the webpage title is correct.<\/p>\n\n\n\n Naturally, in real life we should not use such code \u2013 instead use an assertion<\/strong>. Therefore, now will ask ChatGPT for an assertion.<\/p>\n\n\n\n In the examples above ChatGPT has no problem with indicating libraries, which should be imported for the code to work properly. Still, I was surprised with the fact, that the tool has also managed to prepare the code with the inclusion of assertion in TestNG.<\/p>\n\n\n\n Unfortunately, ChatGPT not always generates response in such a way. In case of a query regarding Robot Framework, the tool did not indicate the necessary library to import.<\/p>\n\n\n\n In this case the SeleniumLibrary should be imported.<\/p>\n\n\n\n In case of Selenium the most frequently asked question concerns the differences between implicit wait and explicit wait. Therefore, it is worth checking if artificial intelligence will handle that issue.<\/p>\n\n\n\n I leave the evaluation of this response to you An excerpt from the documentation on this issue<\/a> may help you with it.<\/p>\n\n\n\n ChatGPT is able to respond to even more complicated queries:<\/p>\n\n\n\n A tester does not live by Selenium alone Nowadays, Cypress is also frequently used and Playwright framework is gaining popularity. Can ChatGPT be helpful in this case?<\/p>\n\n\n\n In case of Cypress the code generated by the tool was working and performed the test correctly:<\/p>\n\n\n Example:<\/p>\n\n\n\n Unfortunately, the situation was not that colorful in case of Playwright. ChatGPT prepared the following code for me:<\/p>\n\n\n Taking into consideration the format, which ChatGPT has adopted, in my opinion the code should look like this:<\/p>\n\n\n Of course, in my research I have also checked other issues concerning automated tests and programming, including queries regarding:<\/p>\n\n\n\n For some time now I have been very interested in the topic of the accessibility. The WCAG standards are connected with that topic.<\/p>\n\n\n\n ChatGPT responds in different languages. For me, the response in Polish was surprising. I also think, that creating web content accessible also for infants is very hard <\/p>\n\n\n\n Of course, also in the case of A11Y (accessibility) we can ask more specific questions.<\/p>\n\n\n\n It should be mentioned here that ChatGPT does not always work as expected. In the example above, I asked the question in English and the answer was given in Polish.<\/p>\n\n\n\n The main goal of the article was to present AI possibilities in the context of testing, however I think that the controversies concerning artificial intelligence are worth mentioning.<\/p>\n\n\n\n Media and authors report, that AI very often uses other people\u2019s work<\/a>. Midjourney creator said to the Forbes magazine:<\/a><\/p>\n\n\n\n There isn\u2019t really a way to get a hundred million images and know where they\u2019re coming from. It would be cool if images had metadata embedded in them about the copyright owner or something. But that’s not a thing; there’s not a registry. There\u2019s no way to find a picture on the Internet, and then automatically trace it to an owner and then have any way of doing anything to authenticate it.<\/em><\/p>\n<\/blockquote>\n\n\n\n From these words, in my opinion, confirmation of the use of other people\u2019s works by artificial intelligence can be deducted. It has a particular significance in the context of the yet another case of right infringement of millions of creators<\/a>, who decided to file a class action lawsuit.<\/p>\n\n\n\n Schools in New York have restricted access to ChatGPT<\/a>. Below are the words of Jenny Lyle, spokesperson for the U.S. Department of Education:<\/p>\n\n\n\n Due to concerns about negative impacts on student learning, and concerns regarding the safety and accuracy of content, access to ChatGPT is restricted on New York City Public Schools\u2019 networks and devices.\u201d<\/em><\/p>\n<\/blockquote>\n\n\n\n While the tool may be able to provide quick and easy answers to questions, it does not build critical-thinking and problem-solving skills, which are essential for academic and lifelong success<\/em>.<\/p>\n<\/blockquote>\n\n\n\n I do agree with these words. I live in a city without access to any sea or lake, and ChatGPT tries to convince me, that there are beautiful beaches in Reda.<\/p>\n\n\n\n Questions arise:<\/p>\n\n\n\n Programs using artificial intelligence still often give wrong answers or provide outdated information.<\/p>\n\n\n\n Above, I have presented and example of the code, that needed to be corrected in order to work, as well as a response, in which ChatGPT proposed ensuring website accessibility to infants.<\/p>\n\n\n\n In my history of using the tool I have also probably encountered a situation of providing outdated information. I asked about differences between Cypress and Playwright. The response was as follows:<\/p>\n\n\n\n Playwright supports a wider range of browsers than Cypress, including Chromium (used by Google Chrome), Firefox, and WebKit (used by Safari). Cypress currently only supports Chromium.<\/p>\n<\/blockquote>\n\n\n\n I decided to look into that response and verify it. On the official website, Cypress informs about the support of many browsers and engines, i.e.: Chrome, Firefox, Edge, Webkit. In this case, we may be dealing with a ChatGPT limitation. ChatGPT contains information until 2021.<\/p>\n\n\n\n Therefore, it is a confirmation, that a human and their control are essential.<\/p>\n\n\n\n Other IT areas are also considering the possibilities of using AI in their work. On YouTube you can find materials on the use of artificial intelligence in the process of creating a website<\/a> or application<\/a>. It is worth looking into what they say.<\/p>\n\n\n\n<\/a><\/a><\/a>Questions on automated tests<\/strong><\/h3>\n\n\n\n
<\/a><\/a><\/a><\/a><\/a><\/a>Question on Robot Framework<\/strong><\/h3>\n\n\n\n
<\/a>Questions on Selenium, Cypress and Playwright<\/strong><\/strong><\/h3>\n\n\n\n
<\/a><\/a>\ndescribe('www.sii.pl', () => {\n it('should open the homepage', () => {\n cy.visit('https:\/\/www.sii.pl');\n \/\/ Assert that the page has loaded successfully\n cy.title().should('eq', 'Strona g\u0142\u00f3wna - SII');\n });\n});\n<\/pre><\/div>\n\n\n<\/a>\nconst { chromium } = require('playwright');\ndescribe('www.sii.pl', () => {\n it('should open the homepage', async () => {\n const browser = await chromium.launch();\n const page = await browser.newPage();\n await page.goto('https:\/\/www.sii.pl');\n \/\/ Assert that the page has loaded successfully\n expect(await page.title()).toEqual('Strona g\u0142\u00f3wna - SII');\n await browser. Close();\n });\n});\n<\/pre><\/div>\n\n\n\nimport { test, expect } from '@playwright\/test'\ntest.describe.only('www.sii.pl', () => {\n test('should open the homepage', async ({ page }) => {\n await page.goto('https:\/\/www.sii.pl');\n \/\/ Assert that the page has loaded successfully\n expect(await page.title()).toEqual('Strona g\u0142\u00f3wna - SII');\n })\n})\n<\/pre><\/div>\n\n\n\n
ChatGPT and the issue of accessibility<\/strong><\/h2>\n\n\n\n
<\/a><\/a>Controversies concerning AI<\/strong><\/strong><\/h2>\n\n\n\n
Copyrights<\/strong><\/strong><\/h3>\n\n\n\n
\n
Education and critical thinking<\/strong><\/strong><\/h3>\n\n\n\n
\n
\n
<\/a>Can AI be relied upon?<\/strong><\/h2>\n\n\n\n
\n
\n
<\/a>