Generate representations for ISAC Knowledge

You might want to get a survey on ISAC's knowledge.

Below is a description how to create XML and HTML representations from the data prepared by math-authors; specific parts of knowledge might be done by a specific selection from the steps below -- such a selection is particularly addressed in the last section.

The manual (and time-consuming !) operations reflect the prototype status of ISAC; automation will come as soon as representation and role of knowledge has settled. The question of representation is related to the open research question about mathematics wikis.

Step 1: Generate XML representation from SML
In order to prepare for manipulations in (1.3) we create the XML-representation in an intermediate step in a temporary directory:

$ mkdir /tmp/xmldata /tmp/xmldata/thy /tmp/xmldata/pbl /tmp/xmldata/met /tmp/xmldata/exp
 * (1.1) create the intermediate target directories, for instance in /tmp/:

$ /usr/local/isabisac/bin/isabelle jedit -l Isac /usr/local/isabisac/test/Tools/isac/Test_Some.thy &
 * (1.2) start the SML development environment with a specific theory Test_Some.thy (the paths might be different; please look at the Installation of the ISAC kernel and ask your administrator):

val path = "/tmp/xmldata/"; thy_hierarchy2file (path ^ "thy/"); thes2file         (path ^ "thy/"); pbl_hierarchy2file (path ^ "pbl/"); pbls2file         (path ^ "pbl/"); met_hierarchy2file (path ^ "met/"); mets2file         (path ^ "met/");
 * (1.3) generate the XML by copying the following lines into an empty ML {* *} block in Test_Some.thy (Test_Some.thy should be rather empty; if not, please delete all non-empty ML {* *} blocks before). Copying immediately executes in an ML {**} block, but it does not in a text {**}; so you might prefer to first copy into the latter kind of block and then replace text by ML. Start with the first line defining the path, which must begin with / :

$ ls -l /tmp/xmldata/*
 * (1.4) check the (many !) files created by the SML functions above, the files for theories in /tmp/xmldata/thy, for problems in /tmp/xmldata/pbl and for methods in /tmp/xmldata/met. Each item of knowledge has its own file, their collective structures are given in thy/thy_hierarchy.xml, pbl/pbl_hierarchy.xml, met/met_hierarchy.xml and exp/exp_hierarchy.xml respectively:

$ cp -r ~/proto4/repos/xmldata/exp/* /tmp/xmldata/exp
 * (1.5) there is no transformation for examples since there exists no SML representation for them; so, for completeness required in step 3, copy these files, most likely from the current installation of the ISAC tutoring system (assumed at /home/USERNAME/proto3/repos/xmldata/exp):

../isabisac$ hg revert test/Tools/isac/Test_Some.thy
 * (1.6) clean up your updates in Test_Some.thy

Step 2: Merge existing content into new XML
ISAC's XML representation is meant to contain all knowledge: knowledge created by math authors plus all knowledge added by course designers. So the *.xml files are under version control: presently they are maintained in the repository of the ISAC front-end. Below we assume this repository at /home/USERNAME/proto4/repos/xmldata.

The *.xml files created by the previous steps contain the knowledge provided by math-authors only, and content added by course designers to respective files needs to be merged into the newly created files. This is done most efficiently using a merge tool; below we assume meld as such a tool:

$ meld /tmp/xmldata/ ~/proto4/repos/xmldata/ &
 * (2.1) compare the newly created files with the versions in the current tip of the repository:

This merge concerns several hundred files already in the prototype and tends to be time-consuming; it is an interesting question how this task could be supported mechanically in the future. Presently there are only a few tricks which might help:

xmldata/pbl$ grep EXPLANATIONS *.xml : pbl_equ_univ_root_sq.xml:   pbl_equ_univ_root.xml:   pbl_equ_univ.xml:   This is the root of automated problem refinement by ISAC. ... pbl_equ_univ.xml: ... in the hierarchy while normalizing the equation.  pbl_equ.xml:   pbl_fun_deriv_nam.xml:   : The above output indicates, that pbl_equ_univ.xml is the only file in the list, which contains  to be copied into the new files.
 * (2.2) detect the files with additional content (which is restricted to the tag EXPLANATIONS), for instance in problems:

The data in the *.xml files are best checked by viewing the generated html format (by the ISAC tutor or by a web browser). Therefore it is a good idea to create the *.html files (step 3 below) and then iterate back to the current step 2.

Step 3: Check XML/HTML syntax of content
Note the specific restrictions on the HTML format below.

The ISAC tutoring system displays knowledge in HTML format, which has been transformed from XML (with HTML inserted in specific tags) by use of style-sheets (see ~/proto4/repos/src/java/isac/util/genhtml/templates/). These transformation needs to be checked for all *.xml files:


 * (3.1) (((TODO: is not 1st))) start the Java development environment and there run KEStore (see installation Pt.(5.1), only run-configuration KEStore.

: GENHTML_DESTINATON_PATH=/tmp/kbase/
 * (3.2) set the paths to /tmp/: we have the *.xml files at /tmp/xmldata/, and we do not want to overwrite the current installation and want to check what actually is being generated. The paths are set in file proto4/repos/src/java/properties/KEStore.properties:
 * 1) XML_DATA_PATH=/home/USERNAME/proto4/repos/xmldata/
 * 1) GENHTML_DESTINATON_PATH=/home/USERNAME/proto4/kbase/

:
 * (3.2.b) If you want to run the ISAC tutor completely on /tmp/ the you also adapt ObjectManager.properties:
 * 1) WEB_PATH=file:///home/USERNAME/proto4/kbase/

$ mkdir /tmp/kbase /tmp/kbase/thy /tmp/kbase/pbl /tmp/kbase/met kbase/exp
 * (3.3) the destination directories must exist, where GENHTML_DESTINATON_PATH points to:

run GenHTML and check the results after a few seconds: $ cd /tmp/kbase $ ls -l
 * (3.4)  generation is done by isac/util/GenHTML.java  with KEStore.properties; first run KEStore (!) from the run configurations, then


 * (3.5) errors caused by generation are reported in /tmp/kbase/GENHTML_error_log.txt. You might prefer to improve and check files one by one; let's assume you work on the single file met_SP_Ztrans_inv_subpbl.xml

FILE_TO_CHECK=met_SP_Ztrans_inv_subpbl.xml GENHTML_DESTINATON_PATH=/tmp/kbase/
 * (3.6) prepare the run configuration GenHTMLsingle defined in file GenHTMLsingle.properties for file met_SP_Ztrans_inv_subpbl.xml):


 * (3.7) generate the HTML format for the single file with run configuration GenHTMLsingle. Errors are reported in file /tmp/kbase/GENHTML_error_log-met_SP_Ztrans_inv_subpbl.txt

cp -r /home/user/USERNAME/proto4/repos/xmldata/thy/img/  /tmp/kbase/thy/ cp -r /home/user/USERNAME/proto4/repos/xmldata/pbl/img/  /tmp/kbase/pbl/ cp -r /home/user/USERNAME/proto4/repos/xmldata/met/img/  /tmp/kbase/met/ cp -r /home/user/USERNAME/proto4/repos/xmldata/exp/img/  /tmp/kbase/exp/
 * (3.8) provide graphics files at the right locations. Present handling graphics of files is particularly cumbersome: Since ISAC displays content in HTML format and the *.html files are in kbase/, graphics files are in kbase/ as well. However, kbase/ is not under version control and graphics files could be easily lost. So graphics files are in the repository repos/xmldata although the system does not need it there. This is confusing when trying to understand the system, and it requires these steps:

Step 4: Generate HTML for displaying content
The ISAC tutor requires HTML format for displaying content, which thus might contain graphics files (and later also movies etc.); this format has been generated already in step 3 (for the purpose of checking). For viewing the content in a standard browser, too, a respective representation for the hierarchies is given by JavaScript.

In step (3.4) the following files for the hierarchies have been generated already in thy/, pbl/, met/ and exp/ respectively: These additional steps make the knowledge browsable by existing JavaScript code

$ cp ~/proto3/repos/xmldata/js-css-auxfiles.zip /tmp/kbase/thy/ $ cp ~/proto3/repos/xmldata/js-css-auxfiles.zip /tmp/kbase/pbl/ $ cp ~/proto3/repos/xmldata/js-css-auxfiles.zip /tmp/kbase/met/ $ cp ~/proto3/repos/xmldata/js-css-auxfiles.zip /tmp/kbase/exp/
 * (4.1) copy style files and JavaScript files required for browsable hierarchies as follows:

$ cd /tmp/kbase/thy/ $ unzip js-css-auxfiles.zip $ cd /tmp/kbase/pbl/ $ unzip js-css-auxfiles.zip $ cd /tmp/kbase/met/ $ unzip js-css-auxfiles.zip $ cd /tmp/kbase/exp/ $ unzip js-css-auxfiles.zip

file:///tmp/kbase/thy/index_thy.html file:///tmp/kbase/pbl/index_pbl.html file:///tmp/kbase/met/index_met.html file:///tmp/kbase/exp/index_exp.html
 * (4.2) now all data can be browsed; just open these urls in your web browser (in 2020 does NOT work anymore -- old software versions ?!?):

Step 5: Move data back to repository and standard installation
An urgent reason not to do step 1 and step 2 on the original data is the fact, that hundreds of files are involved; several development steps involving such a mass of files would affect the efficiency of the version management system, so a rigorous reduction of involving version management is advisable.

In the meanwhile the ISAC tutoring system can run on the data under development in /tmp/xmldata/, since the XML_DATA_PATH has been set accordingly in step (3.2). As soon as the development of the XML representation has been settled, the respective data come into the repository in one go:

$ rm -r /home/USERNAME/proto3/repos/xmldata/* $ cp -r /tmp/xmldata/* /home/USERNAME/proto3/repos/xmldata/
 * (5.1) transfer /tmp/xmldata/ into the version management; we assume the respective location at /home/USERNAME/proto3/repos/xmldata/:

:
 * (5.2) set the configuration in KEStore.properties back to the standard installation with new data:
 * 1) XML_DATA_PATH=/tmp/xmldata/

$ rm -r /home/USERNAME/proto3/kbase/* $ cp /tmp/kbase/* /home/USERNAME/proto3/kbase/
 * (5.3) Finally copy /tmp/kbase/ to the installation (assumed at /home/USERNAME/proto3/kbase/):

:
 * (5.4) If you had directed the ISAC tutor in point (3.2.b) above for reading *.html files from /tmp/kbase/, then redirect to the standard installation in ObjectManager.properties:
 * 1) WEB_PATH=file:/tmp/kbase/

Add or modify single knowledge items
Sometimes just a few items have been added by a math-author, and we want to bypass the labourious step 2. In this case the above setup of step 1 to step 5 is also useful, we re-use the respective numbering below. Let us assume, a math-author has added a new TP-based program ["withSubproblem","Inverse", "Z_Transform", "SignalProcessing"] with the filename met_SP_Ztrans_inv_sub.

val path = "/tmp/xmldata/"; met_hierarchy2file (path ^ "met/"); mets2file         (path ^ "met/");
 * (A)(1.3) in the SML development environment generate the XML from SML by copying the following lines into an empty ML {* *} block in Test_Some.thy:

ls -l /tmp/xmldata/met/met_SP_Ztrans_inv*
 * (B)(1.4) check if the file has been created, there should now be one additional specific file name:


 * (C)(2) add explanations in the EXPLANATIONS tag of the new file: Presently the explanations are in HTML format; explanations must be inserted between   without newlines; the reason for that is easier surveys in step 2, in particular point (2.2).

: GENHTML_DESTINATON_PATH=/tmp/kbase/
 * (D)(3.2) in order to check HTML generation set the paths in KEStore.properties:
 * 1) XML_DATA_PATH=/home/USERNAME/proto3/repos/xmldata/
 * 1) GENHTML_DESTINATON_PATH=/home/USERNAME/proto3/kbase/


 * (E)(3.4) generate the extended hierarchy by running GenHTML (which actually generates all *.html files; don't forget to run KEStore)

FILE_TO_CHECK=met_SP_Ztrans_inv_subpbl.xml GENHTML_DESTINATON_PATH=/home/neuper/tmp/kbase/
 * (F)(3.6) prepare the run configuration GenHTMLsingle defined in file GenHTMLsingle.properties:


 * (G)(3.7) generate the HTML format for the single file with run configuration GenHTMLsingle (while KEStore is already running). Errors are reported in file /tmp/kbase/GENHTML_error_log-met_SP_Ztrans_inv_subpbl.txt


 * (H)(3.8) if the new content contains a graphics file, let's say met_SP_Ztrans_inv_sub.jpg, then copy it into tmp/kbase/met for review.


 * (I) check the HTML representation by opening ''/tmp/kbase/met/index_met.html in a web-browser.

$ cp /tmp/kbase/met/code.html                 ~/proto3/kbase/met $ cp /tmp/kbase/met/met_SP_Ztrans_inv_sub.html ~/proto3/kbase/met $ cp /tmp/kbase/met/met_SP_Ztrans_inv_sub.jpg ~/proto3/kbase/met
 * (J)(5.3) copy the affected *.html files into the standard installation

$ cp /tmp/xmldata/met/met_hierarchy.xml         ~/proto3/repos/xmldata/met/ $ cp /tmp/xmldata/met/met_SP_Ztrans_inv_sub.html ~/proto3/repos/xmldata/met/ $ cp /tmp/xmldata/met/met_SP_Ztrans_inv_sub.jpg ~/proto3/repos/xmldata/met/
 * (K)(5.1) transfer the affected/new files into the version management:

Restrictions on the HTML format
As explained in Extend_ISAC_Knowledge, the HTML representation of Isac's knowledge is used in Browsers as well as in the WindowApplication's minibrowsers for Examples, Theories, Problems and  Methods; the minibrowsers impose specific restrictions from the respective Java classes (Java 1.6):

  while code generated by some HTML-editor, e.g.  creates the misleading Java error The element type "img" must be terminated by the matching end-tag " ".
 * Images are displayed with correct width and heigth by, e.g.
 * Special case thy_isac_Biegelinie.xml: This file does not work with genhtml --- so copy  bu hand from xml to html (and use an old thy_isac_Biegelinie.html as a model).