See also
cosy.com/views/APL94.txt for a glimpse at Arthur`s presentation there .
comp.lang.apl exchanges :
============================: MON.JAN,980105 :============================
From: glasss@mindspring.com (Seymour Glass) 97/12/23 21:18
Subject: Re: What is K? Any References?
whitaker@tiac.net (Orion Whitaker) wrote:
>Sorry to be clueless but... What is K?
>All I learned is that a search for a single letter topic is non-trivial.
>Do you have any pointers to the literature?
>--------------------------------------------------------------------
>Orion C. Whitaker, III, M.D. Anesthesia & Pain Management
>whitaker@tiac.net Massachusetts
Pain management is your line, eh? Try this, which I got from the K people:
Maintenance is provided for an annual fee of 20% of the K
Developer License Fee.
K Developer License Fee - $100,000 per developer.
Henry Rich
- - - - - - - - - - - - - - - - - - - - - - - - -
From: Mike Kent 97/12/24 00:20
Subject: Re: What is K? Any References?
Seymour Glass wrote:
> whitaker@tiac.net (Orion Whitaker) wrote:
...
> >Orion C. Whitaker, III, M.D. Anesthesia & Pain Management
> >whitaker@tiac.net Massachusetts
>
> Pain management is your line, eh? Try this, which I got from the K
> people:
I don't think they're targeting the SOHO/independent-consultant
market. OTOH, I think this may represet a price reduction ...
#Linux000202
============================: WED.FEB,000202 :============================
17:45-19| ArthurWhitney BoF @ LinuxWorld : K & Web-based time series
analysis on massive amounts of data in real time .
|17:50| Natural migration from UNIX in `80s - NT just detour .
Fin : Price Data analysis
Telecom : Call data records , packets . ClickStream for CRM
Billions of rows . Lehman has 35M rows .
Time series capability not practical in SQL .
Price/Performance problem w Linux SMP . - scalability .
Remote administration hell of a lot easier .
Zurich : " NT very stable as long as just using Kx products . "
Linus hardware Choice > NT . Intel only now catching up to RS6000 systems
@ MorganStanley in `80s & early `90s .
Windows 2000 same speed as NT .
Is Linux going to keep up w Drivers , etc .
Linux supports more disk drives - file formats . NT other devices .
Paging : VM ?
64bit ? lin Nt Unix : 2 3 1 |
Use of Op Sys file sys + for K over many other DBs .
TCP/IP : message overhead : Overhead : 4000 2000 msg%s on 50MHz machine
Thru put : 40MB % sec vs 9MB for NT winsock .
File size 2GB 1.5GB Unix 2.7- 3.5 |( no issue w 64b )|
Do own sorting : 3-4M floats % sec .
Context switching - process capacity : RedHat Linux 10x switch speed .
Why corps stick w MS ? | SGI guy : HP sold NT , when that failed , switched
cust to own Unix .
Linux has ISP , university mkt segments .
Not as scalable as Sun or RS6000 .
JoD w drive MS to separate P&Ls - Software $$ will be separate charge
from Hardware .
MS Linux coming ? Seen job offers . | ODBC advanage to MS . v JDBC
AW : Excel been complex but great interface .
MS SOAP interface .
Lehman DB : 40k secuities , 1000 days , 200 attributes |( bonds )|
10M Time series . 40M RDB rows . |( on 248MHz Sun 2 processor )|
40GB DB w 2GB RAM . Sorted by date . Hi hits on 5 of 200 columns .
get fantastic data locality . 300MB columns .
Day end updates :
Oracle has you sign agree not to publish Benchmarks .
OpenLink w have Open Benchmark software in couple of mo .
1E6 updates % sec .
Comparison of 340k positions over yr 400 cols . SyBaseIQ .
Beat Oracle 50 to 1 storing raw data . very fast updates .
Very hi cache hit rate . Ultra2 SCSI 70MB % sec .
100MB % sec .
Telecom ? a# B# switch Billing traffic class .
many many milions of calls @ speed of switch . ( bashful customer )
several M xactions % day . Linux on multiple machines next to Switch .
Mem : buy 4-5x bigest col .
Never swap - sequential writes to disk .
Going parallel in K the lang . |19:0|
I demoed to several people the ability to execute K in CoSy
I`ve recently implemented .
============================: THU.FEB,010201 :============================
|14:41| AW : Relational DB : KDB : Billion trades
1.5B quotes from major exchanges .
fnd `93 . | Millennium Partners .
10M trades & quotes % day .
Telstra : Capacity Monitoring - international phone calls .
5 hrs to design load scripts . T1 would take 3Wks . to take disks frm Mid
town up to Boston .
Ford toras 3hrs : 50mB % sec .
partitioned by date sorted by symbol -
Fast because Inverted . accident of history which called inverted .
select max price from t |>| 65000 | BirkshireHathaway
few million
select where symbol = 'IBM | fast since data sorted by symbol .
added language :
$select from t a , t b , t c |
select count from t where 0 < deltas price
249E6 / upticks
OLTP 1 M xacts % sec % cpu w loging
OLAP several m records % cpu % sec
Reuters
SCSI 30-40MB % sec fast enough .
DollarVolume : Price * Shares
half of time price stays same .
Aud : where results ? now , all results come bach to master .
fast because of col storage , why fast on updates don`t
understan .
DBA can set up so joins precalculates .
SmallTable / anything less than 1GB - fits in memory .
Don`t denormalize .
KDB 40KB K code , K is 130KB .
accumu xacts for second & bulk update to overcome 250 % OpSys speed .
KDB single threaded
|15:29| Did 2B item DB in 1980 . sorting main weapon for spead .
K constant time appends w exponential allocation .