Difference between revisions of "Extension talk:Selenium.php"

From Organic Design wiki
(rm double up, and width attribute)
(→‎In-wiki test format: rm old notes)
Line 5: Line 5:
  
 
== In-wiki test format ==
 
== In-wiki test format ==
−
We want the ability for tests to be maintained within a wiki environment itself. Remember there is a one to many relationship of each test suite file to multiple test files.
 
−
We want to introduce a new tag so that it would provide distinction between selenium tables and other wiki tables. The tag could be called <nowiki><selenium>...</selenium></nowiki> as this is very distinct. The idea is that any selenium tests generated by the selenium-ide should be able to be pasted as is onto a wiki and wrapped with these <nowiki><selenium>...</selenium></nowiki> tags. The tags will strip out any unwanted html content, and bypass the second stage of the wiki parser to allow the html tables to be rendered just like in Selenium. We also need to suppress the wikiparser from parsing wikitext content inside the tables so that edit content is viewed as is, for example a signature should be left as <nowiki>--[[User:Sven|Sven]] 22:42, 6 December 2007 (NZDT)</nowiki>.
 
−
 
−
Creating the <nowiki><selenium>...</selenium></nowiki> tags appears to have bypassed the 2nd phase of the wikiparser and render tables properly as html now --[[User:Sven|Sven]] 14:24, 11 December 2007 (NZDT)
 
−
 
−
 
−
Rather than dividing the tests into two kinds of article (tests and test-suites) I think it may be more convenient to have just a single article which defines all the tests for a given extension or functionality set.
 
−
 
 
{{Note| Ideally we would like to cater for transclusion of individual tests aswell. Optionally a namespace called ''Test:'' could be created specifically for selenium tests. The extension itself will categorize any <nowiki><selenium></nowiki> tag.}}
 
{{Note| Ideally we would like to cater for transclusion of individual tests aswell. Optionally a namespace called ''Test:'' could be created specifically for selenium tests. The extension itself will categorize any <nowiki><selenium></nowiki> tag.}}
  

Revision as of 04:59, 10 January 2008

People.svg This article is a collaborative effort, and as such it is expected that any work done on it should be done with our collaboration principles in mind. Particularly the idea of turn taking.

Wiki collaboration follows the blackboard metaphor where the focus moves amongst the roles working the article. Each work session performed by a role involves becoming familiar with the state which has changed since the last time they worked on it, and then using their expertise to refine any aspects which now need attention. After doing this the role will either consider the job to be complete and notify the other team members, or will see that further feedback from other members is required. Categorisation and workflow tools can aid in the organisation of notifying members that their feedback is required.

The idea of this extension is to bundle selenium up along with the MediaWiki extension and provide a simple means of incorporating tests along with extensions which can be maintained as both wiki articles, and a special page used to manually run the tests. Also we'd like the ability to run tests on a regular schedule and have an email or talk-page based failure notification system. This would allow us to know immediately when changes to code affect extension version functionality when running in various MediaWiki versions.

Taking testing to a further level of organisation would allow us to package up wiki-based organisational systems which exactly specify their functionality and exhibit corresponding tests for every item specified. This would allow us to move into a higher level of service automation and would form the basis of a more robust SLA-based service architecture.

In-wiki test format

Note.svg Note: Ideally we would like to cater for transclusion of individual tests aswell. Optionally a namespace called Test: could be created specifically for selenium tests. The extension itself will categorize any <selenium> tag.


If each test was a separate article, then they can be categorized etc. Utilizing existing Selenium IDE output HTML table format would be convenient. Each test article would be a suite of tests where each individual test is in a separate heading section, and then the actual functions composing the test is just a table within its section. This simplifies things by allowing all the tests for a given extension to be in just one article. It is quickest to maintain HTML tables as these can be imported quickly from the Selenium IDE as is. The only redundant tags are the <html>...</html> wrappers.

It appears that the table tags <thead></thead><tbody></tbody> are not currently defined by the wiki parser, see W:Help:HTML_in_wikitext#Permitted_HTML.

Either use a <selenium> or just add tag-hooks for thead and tbody which return null - probly easiest --Nad 09:19, 7 December 2007 (NZDT)

I guess for simplicity the test environment should utilize the current way selenium works taking you to a URL which shows the selenium frames and the tests it is about to perform where you select the article to test from the Special page.

Selenium TestRunner environment ([PATH to selenium core]/TestRunner.html)

The extension should come with a test that examines the selenium extension, checks it is installed, what version it is and usage.

Section regex

The following example appears to work properly; <php> ?php

$wikitext = <<< EOF

Section one

<selenium> This is some text inside a selenium tag </selenium> This is some more information <selenium> An additional test associated with the first section </selenium>

Section two

Blah blah blah <selenium> inside the second section encapsulated by selenium tags </selenium> EOF;

preg_match_all('|^={2,}\s*(.+?)\s*={2,}(.+?</selenium>)|ms',$wikitext,$sectionNames);

while (count($sectionNames[1])) { 

                       $title= array_shift($sectionNames[1]);
                       print("-------Title: $title--------\n");

                       $content = array_shift($sectionNames[2]);

                       print("-------Content: $content--------\n");
               }

?> </php> This renders the output; 

-------Title: Section one--------
-------Content: 
<selenium>
This is some text inside a selenium tag
</selenium>--------
-------Title: Section two--------
-------Content: 
Blah blah blah
<selenium>
inside the second section encapsulated by selenium tags
</selenium>--------

--Sven 22:37, 20 December 2007 (NZDT)


Specialpage

The url http:///www.organicdesign.co.nz/Specialpage:Selenium should allow users to select an article that they want to test from a pull down list, when the test is run to selenium the suite html will be created on the fly by selecting the url - http://www.organicdesign.co.nz/Special:Selenium/Test article. As the article is made up of individual tests, the special page would also create the test html dynamically, e.g. <bash>

  1. Test article

http://www.organicdesign.co.nz/Special:Selenium/Test_article http://www.organicdesign.co.nz/Special:Selenium/Test1 http://www.organicdesign.co.nz/Special:Selenium/Test2 

http://www.organicdesign.co.nz/Special:Selenium/Test_article#third_selenium_tag_header

http://www.organicdesign.co.nz/Special:Selenium/Test4 ...

  1. Extension location

$IP/extensions/Selenium/Selenium.php

  1. Selenium core location

$IP/extensions/Selenium/selenium-core-0 $IP/extensions/Selenium/selenium -> $IP/extensions/Selenium/selenium-core-0

</bash>

The expected html of a selenium test is currently a test Suite file which links to many tests. An example structure is

Test suite wrapper file
<html><head><meta content="text/html; charset=ISO-8859-1" http-equiv="content-type">
<title>Test Suite</title>
</head><body><table cellpadding="1" cellspacing="1" border="1"><tbody>
<tr><td><b>Flouzo Test Suite</b></td></tr>

<tr><td><a href="/selenium/Wikitests/Logout.html">Logout.html</a></td></tr>

</tbody></table></body></html>
A selenium test - /selenium/Wikitests/Logout.htm
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>New Test</title>
</head>
<body>
<table cellpadding="1" cellspacing="1" border="1">
<thead>
<tr><td rowspan="1" colspan="3">New Test</td></tr>
</thead><tbody>
<tr>
	<td>open</td>
	<td>/mediawiki-1.11.0/index.php?title=Special:Userlogout</td>

	<td></td>
</tr>
</tbody></table>
</body>
</html>

Special page param ignores content after hash

The variable $param does not contain article information after a hash, e.g. http://localhost/SpecialSelenium/SuiteArticle#Test_one, $param equals SuiteArticle. If I access $_REQUEST['title'], it also appears to be SuiteArticle. do you know a hack to grab the entire url path? --Sven

This url (https://organicdesign.nz/files/info.php/foo#bar) shows that the hash doesn't appear in any server variables, so the url's will have to have their hash url-encoded as a %23 (https://organicdesign.nz/files/info.php/foo%23bar). --Nad 22:42, 21 December 2007 (NZDT)
Ok, so in theory using url exncoding should get around this. There is a couple of things I noticed here, firstly the PHP variables available even changes between versions of PHP5, e.g. _SERVER["SCRIPT_URI"] - This will be due to the version of apache rather than php probably. After doing some tests it appears that even if I specify a url encoded hash MediaWiki seems to run a url decode pass over it and strip everything out after the # --Sven 05:43, 22 December 2007 (NZDT)
So either use one of the SERVER vars (probably the PATH_INFO), or change the url format to not use a hash to specify the test, use another slash, or an ampersand or something. --Nad 09:27, 22 December 2007 (NZDT)
After a bit of thought I decided to add a query string parameter, Special:Selenium/Article?section=specific_section, this looks cleaner, correctly specifies the article, and the query string key section is in $_REQUEST which can be checked via;

<php> 

if( array_key_exists('section', $_REQUEST)) { 
  # Process test
} else {
  # Process suite

}


</php>


--Sven 22:10, 23 December 2007 (NZDT)

A good way, but I'd recommend also making the test name into a qs item too so that you're not combining PATH_INFO (the /Article part) with query-string. So you may as well use something like the following URL format:
https://organicdesign.nz/wiki/index.php?title=Special:Selenium&suite=Foo&test=Bar
Yeah good idea, after discussion I agree that suite and test are best for the context of the extension --Sven 09:16, 26 December 2007 (NZDT)

Test article naming

The name of the article should be in the "test", "selenium" or maybe "spec" namespace and have the same title as the extension it is designed to test. This way, the Selenium extension can offer an action button to the tests if there is one corresponding to the current page. Whatever the name, there should be a unique namespace, then these articles could be transcluded into a master article allowing template variable embedding, however we want to suppress certain wikitext though.

Specification

Each section is a test for some specific functionality which should be described clearly in plain english below the heading before the bullet list/HTML table of test functions. This way each extension's test article becomes a complete specification of the extended functionality which is both human readable and machine testable. For this reason the "spec" namespace may be more appropriate than "test".

Test Execution

The special page allows all the known tests to be listed, and specific tests to be executed by launching the Test Runner. The test runner requires the tests to be in a simple HTML table format, and requires them to be available from a URL relative to the Selenium code. The special page must therefore be able to accept for its parameter the name of the tests to run and be able to return them in the appropriate format for Selenium.

Test Automation

We need to include as part of the extension (maybe in an additional PERL script) the ability to run the tests automatically from crontab and report to us via talk page or a test log article any problems which crop up due to code or environment changes.

How does this complement SubVersion (SVN)?

I guess SVN testing can only check server side PHP for errors in core (and extension) MediaWiki functionality, whereas Selenium is testing the output environment from the end user clients perspective.

Specific Tests

See also

MediaWiki:Common.css

The following CSS rules can be added to enhance table rendering; 

/** CSS placed here will be applied to all skins */
div.selenium table {
border:1px solid #CCCCCC;
border-collapse:collapse;
font-family:Verdana,Arial,sans-serif;
font-size:12px;
}
Retrieved from "https://organicdesign.nz/wiki/index.php?title=Extension_talk:Selenium.php&oldid=80662"