RSelenium’s sendKeysToElement command is not working out for you? If so, I may have a solution to offer.
Text boxes are practically everywhere — we use them to google, to log into our online accounts. It’s used to prompt feedback from a webpage based on a keyboard input. Therefore, learning how to interact with them is often crucial to a successful web scraping and automation task. RSelenium’s vignette has a straightforward two-step solution using sendKeysToElement, as summarized below:
Step 1. Locate Textbox in the DOM and assign to WebElem
Step 2. Use command sendKeysToElement(key). The keys are entered as a list.
This is when coding becomes a creative endeavor: what if the text box doesn’t accept keys?
Take my recent project as an example: the task involves extracting data within a specific date range. Manually it’s done using a pop-up calendar as date picker; once selected, the date would appear in the text box in the standard “yyyy-mm-dd” format. Clearly to have RSelenium simulate the actual manual clicks would involve way too many steps. Since the date selection is ultimately entered into a text box, the simpler option would be to save the desired date as an R variable, and then pass this variable as keys to the text box using sendKeysToElement. However, here’s the caveat that fails the latter option: examining the HTML elements, we can see that the text box is inconveniently defaulted to be read-only.
What this means is that the default status prevents the WebElem, ie. the “Start Date” text box, from accepting any keys sent by the sendKeysToElement command. The vignette does not provide a direct solution to this problem; but a different command in the vignette can be used to engineer the solution:
This command allows RSelenium to directly interact with the webpage using JavaScript. You don’t need to be a JavaScript expert: as you can see in this example, the action ‘removeAttribute’ is intended for the argument given in the list of arguments (args). JavaScript, like Python (and unlike R), is zero-based; therefore it starts counting from zero which means argument[0] refers to textbox, the one and only variable in this case.
Upon executing this line in R, the ‘readonly’ attribute is now removed from the HTML of the “Start Date” text box and you’re now free to enter the desired start date!
TL;DR: an additional step is needed after Step 1 (findElement) and before Step 2 (sendKeysToElement): inject JavaScript to remove the “readonly” attribute in order to activate the text box for taking input.
