CRSP

From ISOR
(Difference between revisions)
Jump to: navigation, search
 
(4 intermediate revisions by one user not shown)
Line 1: Line 1:
This page is for CUPL-Tools only. For the Windows desktop application -> [[CRSPSift]].
+
Chicago Booth is going to retire the desktop tool [[CRSPSift]] by end of 2024 and all the other tools based on fileformat 1.0.
==CRSP, COMPUSTAT, CCM ==
+
=== CUPL Tools ===
+
==== access to your remote text terminal ====
+
===== require credentials =====
+
email the VDC sysadmin to provide credentials and get your remote text terminal prepared.
+
      <ul>
+
        <li>
+
            host: crsp.vdc.univie.ac.at
+
        </li>
+
          <li>
+
            user: your username
+
        </li>
+
        <li>
+
            password: your password
+
        </li> 
+
      </ul>
+
  
==== VPN ====
+
Fileformat 2.0 has been recently released and (re)programming on our side has already started accordingly.
If working from home you have to establish a Virtual Private Network (VPN) to Uni first ([https://zid.univie.ac.at/en/vpn/ see ZID/VPN]).
+
 
+
==== SSH ====
+
If working from Windows, you might need to install an SSH-client ([https://www.putty.org/ like PUTTY]) to use Secure Shell (SSH) to connect to your terminal. Accept the server key.
+
[[File:3_accept_key.png|left|frameless|200px|caption]]
+
Also see [https://zid.univie.ac.at/online-speicherplatz/anleitungen/sftp/ ZID's explanation]
+
<br clear=all>
+
 
+
==== usage of a terminal ==== 
+
Now you work remotely and use the remote host's command line (which is the bash shell, don't type the beginning "$", which is the prompt):   
+
[[File:4_terminal_session.gif|left|]]
+
<br clear=all>
+
the midnight-commander (mc) provides a built-in editor mcedit (exit it with Alt+0)
+
<pre>$ mc </pre>
+
<pre>$ mcedit ts_samp7.rqt </pre>
+
e.g. use  '''''ts_print''''' to process a requirementfile '''''ts_samp7.rqt''''' and get the output '''''ts_samp7.out''''':
+
        <pre>$ ts_print ts_samp7.rqt</pre>
+
<!--
+
[[File:7_ts_print.png|left|frameless|500px|caption]]
+
<br clear=all>
+
-->
+
   
+
use mc / mcedit to look into ts_samp7.out:
+
<pre>$ mcedit ts_samp7.out </pre>
+
[[File:8_ts_samp6_out.png||frameless|400px|caption]]
+
<br clear=all>
+
<br/>
+
use e.g. this command to get the switches for the command '''''crsp_print''''':
+
<pre>$ crsp_print /d1  /var/crsp/2019-07/crspdata/cmz201807</pre>
+
 
+
==== download the results onto your desktop / laptop ====
+
use an SFTP client like [https://cyberduck.io/download/ CyberDuck] on a Mac or  [https://zid.univie.ac.at/online-speicherplatz/anleitungen/sftp/ WinScp] on Windows to download the results (e.g. ts_samp7.out) onto your desktop for further processing. Provide your SSH credentials:
+
 
+
[[File:10_cyberduck_credentials.png|400px]]
+
<br clear=all>
+
 
+
==== documentation====
+
Please read some manuals to gain experience with the ts_sampleX.rqt files in your home directory
+
<ul>
+
<li>
+
[http://www.crsp.org/products/documentation/cupl-386-guide CRSP Utilities and Program Libraries (CUPL)]
+
</li>
+
<li>
+
[https://w3.loibl.com/uni/ccm_data_guide_0.pdf CCM Data Guide]
+
</li>
+
<li>
+
and maybe the [https://w3.loibl.com/uni/xf_understanding_the_data.pdf xf-understanding_the_data guide].
+
</li>
+
<li>
+
collected [https://w3.loibl.com/wiki/index.php/CRSP#Usage thoughts] .
+
</li>
+
</ul>
+
=== WEB Access R ===
+
* apply for login-credentials andreas.loibl@univie.ac.at
+
* use sftp to upload / download your date (like with your [https://zid.univie.ac.at/online-speicherplatz/anleitungen/sftp/ Online-Speicherplatz])
+
* http://calc.isor.univie.ac.at:8787
+
* explore some commands, for the sake of functionality only (not to make any sense)
+
###### Example#1
+
# search for "EGSA"and perform some actions
+
# comment, not interpreted
+
? system
+
? system.time()
+
help("crsp.data.usstock")      ## we have licensed US-Stock 1925 and  4 US-Indices and CCM
+
system2("uptime")              ## load average < 10 ! (otherwise too many concurrent users)
+
library(data.table, crsp)
+
setwd('~/crsp')
+
getwd()
+
myheaders <- data.table(crsp.data.usstock('sfz_hdr'))
+
myEGASHeaders <- myheaders[HTSYMBOL %like% "EGAS" & BEGDT > "1986-01-8"]$KYPERMNO
+
myMonthly <- crsp.data.usstock("sfz_mth")
+
nrow(myMonthly)
+
ncol(myMonthly)
+
head(myMonthly,10)
+
MyEGASData <- myMonthly[myMonthly$KYPERMNO %in%  c(myEGASHeaders),]
+
head(MyEGASData)
+
tail(MyEGASData)
+
plot.ts(MyEGASData$MPRC)
+
######## Example #2
+
# read tickers from one file perform some operations and write results to another file
+
# myTechies <- read.csv('/home/su/crsp/techcompanies.tickers', header = FALSE)
+
myTechies <- scan(file = 'techcompanies.tickers', what = 'character', sep = ',')
+
myTechiesHeaders <- myheaders[myheaders$HTSYMBOL %in% c(myTechies)]
+
#fileDsk <- tempfile("MyTechiesHeaders")
+
fileDsk <- "MyTechiesHeaders"
+
#setwd("~/R_workspace/from_uspace/privat/crsp_sample1/")
+
write.table(t(myTechiesHeaders), file='./MyTechiesHeaders')
+
# file.show(fileDsk)
+
# if(interactive()) file.show(fileDsk)
+
# unlink(fileDsk) # delete file again
+
myTechiesData <- myMonthly[myMonthly$KYPERMNO %in%  c(myTechiesHeaders$KYPERMNO),]
+
plot.ts(log(myTechiesData$MPRC))
+
 
+
 
+
==== Documentation ====
+
[http://www.crsp.org/products/documentation/overview-4 in the crsp.org universe.]
+

Latest revision as of 14:16, 27 September 2023

Chicago Booth is going to retire the desktop tool CRSPSift by end of 2024 and all the other tools based on fileformat 1.0.

Fileformat 2.0 has been recently released and (re)programming on our side has already started accordingly.

Personal tools
Namespaces

Variants
Actions
Navigation
Tools