ctakes-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mullane, Sean *HS" <SP...@hscmail.mcc.virginia.edu>
Subject RE: YTEX setup notes
Date Mon, 13 Feb 2017 16:25:57 GMT
Our main interest in YTEX is the database reader and consumer that it comes with. That is probably
a smaller subset of its functionality but it was a good source for those. We are still working
on implementation, but we plan to run an annotation pipeline on the new notes we get each
day and store the resulting annotations in our database. The notes will be read from a database,
parceled out in chunks to several processes, annotated, and consumed by the database consumer.

Sean

-----Original Message-----
From: Dunlop, Joyce (HP) [mailto:Joyce.Dunlop@va.gov] 
Sent: Monday, February 13, 2017 11:17 AM
To: Mullane, Sean *HS
Cc: dev@ctakes.apache.org
Subject: RE: YTEX setup notes

Hi Sean,

I will have to check with the VA Researchers to find out what they planned to for the ytex
web. 
So you don't use YTEX web interface but you do use ytex correct?  Could you share the steps
in a typical process you run for YTEX?

Thanks,
Joyce

-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu] 
Sent: Monday, February 13, 2017 7:59 AM
To: Dunlop, Joyce (HP)
Cc: dev@ctakes.apache.org
Subject: [EXTERNAL] RE: YTEX setup notes

Hi Joyce,

I wound up solving the error by commenting out the bean definition in ctakes-ytex-res/src/main/resources/org/apache/ctakes/beans-kernel.xml
and ctakes-ytex-res/target/classes/org/apache/ctakes/beans-kernel.xml, which resolved that
error. I think it was deprecated anyway. 

I did run into another error where I didn't have LookupDesc_SNOMED.xml in the directory path.
Added it from the ctakes-src files solved that problem and got it working on Java 8 for us.

I haven't tried using the web interface - we don't have any plans to use it. Can I ask what
you want to use it for? I'm not really familiar with what it does.

Thanks,
Sean

-----Original Message-----
From: Dunlop, Joyce (HP) [mailto:Joyce.Dunlop@va.gov]
Sent: Friday, February 10, 2017 3:57 PM
To: Mullane, Sean *HS
Subject: RE: YTEX setup notes

Hi Sean,

We had to change the return type from BagOfWordsExporter to WekaBagOfWordsExporter in that
file.

From:
	public BagOfWordsExporter getBagOfWordsExporter() {
		return bagOfWordsExporter;
	}

To:
	public WekaBagOfWordsExporter getBagOfWordsExporter() {
		return bagOfWordsExporter;
	}

We have still not be able to get the YTEX web interface to work.  The YTEX application starts
up and we are able to access through the URL:  http://localhost:8080/semanticSim.jsf - We
see 4 tabs but nothing seems functional.  Is there something we are still missing?  We found
references to rest services - are they called separate from the YTEX UI? 

Any clue would be appreciated.

Thanks,
Joyce

PS:  I think there was one other code change to get Java 8 working, so if you get another
error just shoot it over and I should have the answer.






-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Monday, February 06, 2017 3:33 PM
To: Dunlop, Joyce (HP)
Cc: dev@ctakes.apache.org
Subject: [EXTERNAL] RE: YTEX setup notes

Joyce,

I haven't been able to find further documentation. It seems it is somewhat out of date and
some of the Yale sources have been taken down.

In your feedback on my YTEX setup notes, I'm finding that when I try to run YTEX using Java
8 I get an error related to org.apache.ctakes.ytex.weka.GramMatrixExporterImpl. When I run
it on Java 7 I get no errors. You mentioned that upgrading to Spring 3.2 solved an issue for
you. How did you do that - did you change the Maven build file to pull in the newer version
or did you do something else to get the newer Spring version? I'm hoping that will resolve
the Java 8 issue here.

Thanks,
Sean

-----Original Message-----
From: Dunlop, Joyce (HP) [mailto:Joyce.Dunlop@va.gov]
Sent: Friday, February 03, 2017 4:26 PM
To: Mullane, Sean *HS
Cc: dev@ctakes.apache.org
Subject: RE: YTEX setup notes

Sean/others,

Do you by chance have any of the documentation produced by Yale tucked away locally?

I see quite a few dead links that seem to go back to demo's and documents on cTakes site.

Examples:
The demo similarity web app is available under http://informatics.med.yale.edu/ytex.web;

http://ytex.googlecode.com/svn/trunk/projects/nlm.wsd/doc/YTEX%20Semantic%20Similarity%20Measures.pdf

Thanks,
Joyce

-----Original Message-----
From: Dunlop, Joyce (HP)
Sent: Monday, January 30, 2017 8:27 PM
To: 'Mullane, Sean *HS'; dev@ctakes.apache.org
Subject: RE: YTEX setup notes

Sean M,
This is perfect!  

Thanks,
Joyce

-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Monday, January 30, 2017 3:06 PM
To: Dunlop, Joyce (HP); dev@ctakes.apache.org
Subject: [EXTERNAL] RE: YTEX setup notes

Joyce,

Thanks for the notes. I'm glad to see you had success using Java 8 - I'll have to try that.

To answer your question about table contents, see below for the rowcounts in the YTEX tables
after in my installation. Apologies in advance for the formatting:

tablename	tot_cnt
anno_base		0
anno_base_sequence	1
anno_contain		0
anno_date		0
anno_link		0
anno_markable	0
anno_med_event	0
anno_mm_acronym	0
anno_mm_candidate	0
anno_mm_cuiconcept	0
anno_mm_negation	0
anno_mm_utterance	0
anno_named_entity	0
anno_ontology_concept	0
anno_segment		0
anno_sentence	0
anno_token		0
anno_treebank_node	0
classifier_eval		0
classifier_eval_ir	0
classifier_eval_semil	0
classifier_eval_svm	0
classifier_instance_eval	0
classifier_instance_eval_prob	0
corpus_doc		0
corpus_label		0
cv_best_svm		0
cv_fold			0
cv_fold_instance	0
document		0
feature_eval		0
feature_parchd	0
feature_rank		0
fracture_demo		31
hibernate_sequences	1
hotspot		0
hotspot_instance	0
hotspot_sentence	0
kernel_eval		0
kernel_eval_instance	0
ref_named_entity_regex	0
ref_segment_regex	0
ref_stopword		360
ref_uima_type		38
tfidf_doclength	0
umls_aui_fword	6902003
v_snomed_fword_lookup	1374810

Sean

-----Original Message-----
From: Dunlop, Joyce (HP) [mailto:Joyce.Dunlop@va.gov]
Sent: Friday, January 27, 2017 10:12 AM
To: dev@ctakes.apache.org
Subject: YTEX setup notes

Hi Sean M, 

Notes from :
Step
-The Needed to add sqljdbc4.jar to CTAKES_HOME/lib probably need to happen after -3. Install
cTAKES "as usual"

-Java 7 is throwing errors because Spring needs to be upgraded to at least 3.2.
--It looks like Spring was previously upgraded from 2.5 to 3.1, but spring context files -
 xsi:schemaLocation were not previous upgraded and remained 2.5.

-We have been able to successful run the script with JAVA 8 after upgrading Spring to 3.2.18-Release
from the 3.1/2.5 version.   We also upgraded the version of Jetty.

-Step 7 which I think you list as 8 below to execute the setup script. It looks like it has
mostly been successful but I do have a few errors at the bottom related to the database password
we chose which had an & (AMP) character is being parsed on the & character.

3 tables that are showing errors related to the amp characters:
-ref_uima_type.xml
-fracture_demo.xml
-stopword.xml

2 tables are the only table will significant record counts:
-umls_aui_fword
-v_snomed_fword_lookup  

2 other table seem to have one row inserted.

The question I have is when this step completes how many of the tables should have data? 

Thanks again.
Joyce




-----Original Message-----
From: Dunlop, Joyce (HP) [mailto:Joyce.Dunlop@va.gov]
Sent: Tuesday, January 17, 2017 12:33 PM
To: dev@ctakes.apache.org
Subject: [EXTERNAL] RE: New to CTAKES 

Sean M,

I will try to work through the process this tomorrow.

Thanks!

Joyce

-----Original Message-----
From: Mullane, Sean *HS [mailto:SPM9R@hscmail.mcc.virginia.edu]
Sent: Tuesday, January 17, 2017 11:10 AM
To: dev@ctakes.apache.org
Subject: [EXTERNAL] RE: New to CTAKES 

Joyce,

This may help, here are some notes I put together when installing YTEX in our Windows/SQL
Server test environment. I didn't build from source here, this is just installation (including
UMLS). I'm working on writing up these instructions more thoroughly so if you use this please
let me know if you find any points of improvement.

Requirements and steps to set up YTEX
These notes refer to the installation instructions at https://cwiki.apache.org/confluence/display/CTAKES/YTEX+Installation
1. They suggest installing UMLS in our database
	Follow instructions here to install UMLS: https://cwiki.apache.org/confluence/display/CTAKES/UMLS+MS+SQL+Server+Installation
	-http://groups.csail.mit.edu/medg/projects/text/Load_UMLS_mysql.html is useful when installing
from .zip file. The mmsys.zip file needs to be unzipped to access the run64.bat file but also
needs to be present as a .zip file for the run64.bat file to read from.
2. Create a database user and schema for use with YTEX 3. Load UMLS into database
	-Needed to create ytex.properties file from examples files in the developer/source version
of cTAKES
	-Needed to change path in ctakes.profile to match version name of CTAKES_HOME directory
	-Needed to add sqljdbc4.jar to CTAKES_HOME/lib and sqljdbc_auth.dll to C:\Program Files\Java\jre_Version\bin
	-Needed to open port 1433 for TCP/IP traffic and clear dynamic ports
	-Need to change table definitions as follows to install UMLS-2016AB:
		MRCONSO.CODE varchar(75) -- was varchar(50)
		MRDEF.DEF varchar(max) -- was varchar(4000)
		MRSAB.VSAB varchar(24) -- was varchar(20)
		MRSAT.ATN varchar(64) -- was varchar(50)
		MRSAT.ATV varchar(max) -- was varchar(4000) 3. Install cTAKES "as usual"
4. Download and unzip ctakes-ytex-lib-3.1.2-SNAPSHOT.zip 'over' your installation.
	-I believe this step is deprecated - skip to step 6 5. Unzip YTEX Resources (Optional - UTS
login required)
	-I believe this step is deprecated - skip to step 6
	-may be needed for concept graphs and dictionary lookup table for NER 6. Fix the path references
in shell script to match your environment 7. Create CTAKES_HOME\resources\org\apache\ctakes\ytex\ytex.properties
and specify database connection parameters.
8. Execute the setup script
	-Need to use Java 7 - Java 8 errors out
	-If SQL errors encountered, appenders can be added to log4j.xml to collect SQL queries

Sean

-----Original Message-----
From: Finan, Sean [mailto:Sean.Finan@childrens.harvard.edu]
Sent: Tuesday, January 17, 2017 11:57 AM
To: dev@ctakes.apache.org
Subject: RE: New to CTAKES 

Hi Joyce,

If you are building from source then you should not need to manually download the resources.
 Maven should be doing it for you.  Well, that is the behavior of 3.2.3 ... I honestly cannot
remember what 3.2.2 did ...

Otherwise, I think that if the latest was the 3.2.1.1 then that is probably the most appropriate
for the 3.2.2 release if you want all of the resources.

As for building and deploying ytex, I don't have any advice.  Perhaps some ytex power-user
out there can help.

Sean

-----Original Message-----
From: Dunlop, Joyce (HP) [mailto:Joyce.Dunlop@va.gov]
Sent: Tuesday, January 17, 2017 11:25 AM
To: dev@ctakes.apache.org
Cc: Dorner, Andrew J. (PSI); Rustrian, Armando (Liberty ITS)
Subject: New to CTAKES 

Good Morning,

I am trying to set up a development environment using the source release of 3.2.2.

Reading though the documentation on

https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_CTAKES_cTAKES-2B3.2-2BDeveloper-2BInstall-2BGuide&d=DwIFAg&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=yBXENVQKpWjVraf6Zf7uY5l9LJxxrRiiE-yjyFID6d8&s=iqpkHc0kT5mucNnxYyc1mczXXlbmSVJlX-8dxeJvp2o&e=
.

Merge the version-matching resources ZIP file from https://urldefense.proofpoint.com/v2/url?u=http-3A__sourceforge.net_projects_ctakesresources_files_&d=DwIFAg&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=yBXENVQKpWjVraf6Zf7uY5l9LJxxrRiiE-yjyFID6d8&s=wPUG8d9qpl_kQBPP5xI9y84mwMEXfaB2cdbkHvWaa0Y&e=
 into your ctakes-dictionary-lookup-res project.

ctakes-resources-3.2.1.1-bin.zip<https://urldefense.proofpoint.com/v2/url?u=https-3A__sourceforge.net_projects_ctakesresources_files_ctakes-2Dresources-2D3.2.1.1-2Dbin.zip_download&d=DwIFAg&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=yBXENVQKpWjVraf6Zf7uY5l9LJxxrRiiE-yjyFID6d8&s=pErHE2p1qFyPeMACLDvshPiTpNZXM847MgGESOtvyuU&e=
> is available for download.  Is there a 3.2.2 version of the resources?

After reading some of the posts from the email box, and reading the documentation on the site
above.  It looks like the documentation may be somewhat dated and not fulling integrated for
YTEX.

Would you have any recommendations in advance on how to set up a build process for deployment
and YTEX database from scratch?

Thanks,
Joyce












Mime
View raw message