Posts

Left of X, X and Right of X Data Design and Analysis - Using ELK / Elasticsearch / Logstash / Kibana

Image
Explanation below; Map Pictures - Shared Website Snapshots for colleagues operating at farms. Farms have an international scope therefore their data design and analysis infrastructure needs to be able to span global view. Elasticsearch / Kibana / Logstash (ELK Stack) purpose built for internet cyber security originally.  The ELK stack is currently utilized across many domains as a data storage layer for general analysis purposes able to store petabytes level, if need be.  I have met a research group that operates on farms located in NWS Australia. NSW Australia teams have developed regional soil and land use "WMS" - Web Mapping Systems can be linked into Elasticsearch.  Data is owned by the government; there are processes in place for entities to access specific reports that are held in backend databases and PDF reports.  The ELK stack supports adding private data layers on top of public data sources that reference specific places. I have added a couple that referenc...
Post a Comment
Read more

Portfolio Derivative Analysis

Image
  Tick data analysis indicating consistent non-linear stock market correlations across multiple days: Gaps indicate opportunity, as stocks are moving opposite -  delta in gap is possible profit  if you short one side and buy high side. Same stock pairs, similar correlation patterns.: Correlated Stocks are chosen by finding most similar profiles in large graphs. This technique can be applied to many different types of fintech systems: stocks, derivatives, crypto. forex and other funds,  Derivatives create co-associated graphs correlated and evolving across time.  In the graphs below the metric is normalized logarithmic (0.0-1.0) generated every three seconds.  The underlying infrastructure combines EMQX, an MQTT broker capable of sustaining 100 M connections fed into a HPC / MPI streaming infrastructure capable of running 100 M active cooperative agent processes coordinated by its own scheduler.  it has been shown to scale up to handle 5000 financial st...
Post a Comment
Read more

Alternative Data Structures and Metrics Spaces- Enables Better Insights Faster - In Turn Increasing Revenue Growth

Image
  Tick data analysis indicating consistent non-linear stock market correlations across multiple days: Gaps indicate opportunity, as stocks are moving opposite -  delta in gap is possible profit  if you short one side and buy high side. Another segment of time, with same stock pairs: Correlated Stocks are chosen by finding most similar profiles in large graphs. This technique can be applied to many different types of systems; see below. Slides Presented by Dr. Paul Hoffman in Conference Lectures: Early deep learning methods with 100 M features with high performance diagnostics, above. Below is technical method to quickly find similar profiles across millions of attributes from compressed graph sparse matrices. Conceptual unfolding of large graph segment across multiple sparse matrices:  Saffron Tech's initial patent implementation occurred in early 2000's. New uniquely capable parallel processing systems exist that can generate the associative graphs in real-time,  r...
Post a Comment
Read more

Reducing Costs - Optimizing Value - Suggest Using Alternate Methods

Image
 Degraded systems cost directly related to "quality escape" reduces revenues and profits; identifying degradation early allows for scheduled interruptions minimizing disruptions to production.  Non-redundant critical part failure induce safety incidents. Systems modeling, simulation, and analysis traditionally utilize linear correlation methods. Summary if you look at a "cloud" - in linear world you only see a square or a rectangle, all interesting features are removed.   Due to simplifying assumptions many systems break unexpectedly disrupting production introducing additional costs in re-planning, scheduling, personnel, tools and parts acquisition.  Alternate methods increase visibility into root cause methods, in turn, increasing Reliability, Safety, in turn decreasing revenue generation disruptions.   Non linear metrics enable better visibility into system dynamics and degradation patterns. For instance in the aviation domain:  Jet Engine Turb...
Post a Comment
Read more

Precision Ag - Open Source Locally Controlled Network Fabric - SMS, Cell, RF, WRT - Derived from MIT's CSAIL Roof-net

Image
  Think Precision Ag - Farmer Local Control Bypass to the Internet https://telecominfraproject.com/   Derived from other incremental experiments in other areas: 1) Legacy - MIT's CSAIL Roofnet (mid 2000's era) - Mesh Adhoc Networks, as described in this thesis:  https://www.cs.cmu.edu/~dga/15-744/S07/papers/roofnet-mobicom2005.pdf   2) Evolved into OpenWRT - a mature supported version of early Roofnet,  3) Open Source - Software Defined Radio - Evolved from OpenWRT https://www.nuand.com/ Next gen SDR chipsets; enable lightweight embedded projects like drone, land vehicle operational and field sensor environments to be developed - think precision agriculture end-to-end architecture. 4) Open Source BTS -  Integrated Radios with Cell / Internet Access http://openbts.org/about/ OpenBTS integrations with Cellular and SIP  Integration - bridges to the internet. OpenBTS-4.0-Manual.pdf - pg 17 Figure Below In Progress - Open Source Cell Network https://techblo...
Post a Comment
Read more

Recent Stock Market Pictures & Work in Other Domains

Image
Summary Points: 1)  Actionable Domain Insights Generated       2)  By Changes in Architecture, Software, Data, Metrics A Typical View of Stock Value at 30 Second Blocks - Linear  Metric - Averaging Same Collection of Stocks at Finer Level Detail Using Non-Linear Similarity Profile Metrics Collection of Stocks Found to be Non-Linearly Correlated  Finer Level Details in Smaller Time Range Above; Below is larger Time Span with Same Stocks Domain Level Insight Into Market Movement Creating Actionable Opportunity Below A Very Small Timeline Segment of Stock trades within a 1 second band Common Practices Average 1 second values - erasing true Insight  Other Domains Also Average Data Sets Below are Pharma Genomic Interaction Correlations Non Linear Metrics in Upper Triangle - Linear in Lower Right Triangles Deeper Implication is that world is missing opportunities  Other Metrics Exist Providing Better Insight and Efficiency -------------------------...
Post a Comment
Read more

Associate List - In Alphanumeric Ordering - Give people credit, they shared - we learned - enhanced our knowledge base

Many Others not listed here for privacy reasons Arnold, Greg - https://www.linkedin.com/in/william-greg-arnold-1007676/ Bahill, Terry - https://www.linkedin.com/in/terry-bahill-b171a3/ Barnes, Mike - Boeing Manager - Always looking into future in Energetic Manner Benson, Chris - https://www.linkedin.com/in/christopher-p-benson/ Bird, Jeff - https://www.linkedin.com/in/jeff-bird-23a62118a/  Bowen, Don - PhD Acoustic Penn State, only person at Bell Labs I knew with Acoustics Degree Cates, Sol - https://www.linkedin.com/in/solcates/  Chamis, Chrysanthie - https://www.linkedin.com/in/chrysanthie-chamis-874a8b14/ Chen, Henry -  https://www.linkedin.com/in/henry-chen-3109712a/ Cuspard, Steve - https://www.linkedin.com/in/steve-cuspard-41452853/ Donaghey, Jules - https://www.linkedin.com/in/jules-donaghey-621133149/ Eaves, Don - https://www.linkedin.com/in/donaldseaves/ Endzell, Alex - https://www.linkedin.com/in/alex-endzell/ Fi...
Post a Comment
Read more