Devops Monk

Most stock screeners cost $30–$200 per month. Bloomberg Terminal costs $24,000 per year. I built something that does a meaningful fraction of what those tools do — analysing 220+ UK and US stocks every hour, scoring them across six dimensions, detecting bearish warning signals, running insider trading checks via SEC EDGAR, and presenting everything in a React PWA — at zero ongoing cost.

The platform is live at share.devops-monk.com. The full source is at github.com/devops-monk/share-market.

This post is a deep technical walkthrough — how each layer works, why I made each design decision, and how you can run your own instance.

Architecture Overview

The system has three completely decoupled layers. They communicate only through JSON files committed to the GitHub repository.

flowchart TD
    subgraph Sources[Data Sources - Free APIs]
        YF[Yahoo Finance v8\nOHLCV + fundamentals]
        FV[FinViz scraping\nP/E beta earnings US stocks]
        GN[Google News RSS\nHeadlines per stock]
        EDGAR[SEC EDGAR API\nInsider trades US stocks]
        HF[HuggingFace Inference\nFinBERT sentiment model]
    end

    subgraph ETL[ETL Pipeline - Node.js on GitHub Actions]
        FETCH[Fetch and normalise\nall data sources]
        TECH[Compute technical\nindicators]
        SIGNALS[Detect bullish\nand bearish signals]
        SCORE[Composite scorer\n0-100 per stock]
        PRED[Predictive score\nlinear regression]
        WRITE[Write JSON files\nto data/]
        FETCH --> TECH
        TECH --> SIGNALS
        SIGNALS --> SCORE
        SCORE --> PRED
        PRED --> WRITE
    end

    subgraph DASH[Dashboard - React PWA on GitHub Pages]
        OVERVIEW[Overview page]
        SCREENER[Screener with filters]
        BEARISH[Bearish alerts]
        DETAIL[Stock detail page]
        COPILOT[AI Copilot chat]
        PAPER[Paper trading journal]
        MACRO[Macro dashboard]
    end

    Sources --> FETCH
    WRITE -->|JSON committed to repo| DASH
    GH[GitHub Actions\nhourly schedule] --> ETL
    GH -->|auto deploy| DASH

Why this architecture?

Everything is static. No database, no server, no infrastructure to maintain. The ETL pipeline runs on GitHub Actions (free for public repos), writes JSON files back to the repo, which triggers a GitHub Pages deploy. The dashboard fetches those JSON files at page load. The only “runtime” cost is the GitHub Actions minutes — well within the free tier.

Part 1: The ETL Pipeline

GitHub Actions Schedule

The pipeline runs every hour on weekdays, timed to cover both UK (opens 8am BST) and US (opens 2:30pm BST) market hours:

on:
  schedule:
    - cron: '0 7-21 * * 1-5'   # Every hour, Mon-Fri, 7am-9pm UTC
  workflow_dispatch:             # Also triggerable manually

Each run processes all 220 stocks in parallel with controlled concurrency — too many concurrent requests gets the IP rate-limited.

Yahoo Finance: 25 concurrent requests  (tolerant, high throughput)
FinViz:         8 concurrent requests  (aggressive rate limiting)
News fetching: 10 concurrent requests
EDGAR:          5 concurrent requests  (SEC requirement)

Data Sources

flowchart LR
    subgraph Free[Zero-Cost Data Sources]
        YF["Yahoo Finance v8\n/v8/finance/chart/:ticker\nNo auth required\nOHLCV 5yr daily\n52w range, volume"]
        FV["FinViz\nHTML scraping\nUS stocks only\nP/E forward P/E\nearnings growth\nrevenue growth beta"]
        GN["Google News RSS\nPer-stock headlines\nLast 7 days\nFree no auth"]
        ED["SEC EDGAR API\n/submissions/:cik\nInsider Form 4 filings\n90-day net buys/sells"]
        HF["HuggingFace\nFinBERT model\nSentiment scoring\nFree tier"]
    end

Why Yahoo Finance v8? It is the only truly free source for OHLCV data with 5 years of history. The v8 chart endpoint is the same one used by the Yahoo Finance website — it requires no authentication and returns JSON directly.

Why scrape FinViz instead of an API? FinViz’s API costs $40/month. Their website is public. I scrape it with respectful rate limits (8 concurrent, 50ms delays) and a descriptive User-Agent. This gives P/E, forward P/E, earnings growth, revenue growth, and beta for US stocks.

Why SEC EDGAR for insider trades? The SEC requires all insider transactions (Form 4 filings) to be publicly disclosed within 2 business days. EDGAR provides a free JSON API to query these. When insiders buy their own stock, it is often a meaningful signal — they know the company better than anyone.

Part 2: Technical Indicators

The pipeline fetches 5 years of daily OHLCV data (approximately 1,260 trading days) and computes the following indicators for each stock:

Trend Indicators

Momentum Indicators

Volatility and Volume

Candlestick Patterns

The pipeline also detects 14 candlestick reversal patterns:

• Bullish: Hammer, Bullish Engulfing, Morning Star, Bullish Harami, Bullish Marubozu, Doji (at support)
• Bearish: Shooting Star, Bearish Engulfing, Evening Star, Bearish Harami, Bearish Marubozu

Part 3: Signal Detection

Indicators are combined into discrete signals with direction (bullish/bearish), severity (1–3), and timeframe (short/medium/long):

flowchart TD
    subgraph Long[Long-term signals]
        GC[Golden Cross\nSMA50 above SMA200\nbullish severity 2]
        DC[Death Cross\nSMA50 below SMA200\nbearish severity 3]
        MAA[MA Alignment\nPrice SMA50 SMA200 aligned\nbullish or bearish severity 1]
    end
    subgraph Medium[Medium-term signals]
        MACDB[MACD Bullish\nhistogram positive]
        MACDD[MACD Bearish\nhistogram negative]
        ADXT[ADX Strong Trend\nADX above 25]
        ICHI[Ichimoku Cloud\nabove or below cloud]
    end
    subgraph Short[Short-term signals]
        RSIO[RSI Overbought\nRSI above 70]
        RSIOV[RSI Oversold\nRSI below 30]
        STOB[Stochastic Bearish\ncrossover in overbought]
        STBU[Stochastic Bullish\ncrossover in oversold]
        BBU[BB Upper plus RSI High\noverextended]
        BBL[BB Lower plus RSI Low\npotential bounce]
        BBS[BB Squeeze\nbreakout imminent]
        OBVD[OBV Divergence\nvolume not confirming price]
        CAND[Candlestick patterns\n14 reversal patterns]
    end

    Long --> Score[Signal Score\nbullish severity minus bearish severity]
    Medium --> Score
    Short --> Score

How signals become a score:

const bullishWeight = signals
  .filter(s => s.direction === 'bullish')
  .reduce((sum, s) => sum + s.severity, 0);

const bearishWeight = signals
  .filter(s => s.direction === 'bearish')
  .reduce((sum, s) => sum + s.severity, 0);

const netSignal = bullishWeight - bearishWeight; // typically -10 to +10
const technicalSignals = normalize(netSignal, -8, 8); // maps to 0-100

Severity 3 signals (Death Cross, RSI + Stochastic confluence) carry three times the weight of severity 1 signals. This prevents a cluster of weak signals from outweighing one strong one.

Part 4: The Composite Scoring Algorithm

Every stock gets a 0–100 composite score built from six normalised sub-scores:

flowchart LR
    subgraph Inputs[Raw Data]
        R3[3-month return]
        R6[6-month return]
        SIG[Bullish minus bearish\nsignal weight]
        SENT[FinBERT sentiment\n-1 to plus 1]
        PE[P/E ratio]
        EG[Earnings growth]
        VR[Volume ratio\nvs 20-day avg]
        BETA[Beta]
        VOL[30-day volatility]
    end

    subgraph SubScores[Normalised Sub-scores 0-100]
        PM[Price Momentum\navg of 3m and 6m returns\nnorm -50pct to plus 50pct]
        TS[Technical Signals\nnet signal weight\nnorm -8 to plus 8]
        NS[News Sentiment\nnorm -1 to plus 1]
        FU[Fundamentals\ninverted P/E plus growth\nblended 50 50]
        VT[Volume Trend\nnorm 0.3x to 3x avg]
        RI[Risk Inverse\n100 minus blended beta and volatility]
    end

    subgraph Composite[Weighted Composite]
        C[Score 0-100\nPM times 0.25 plus TS times 0.25 plus\nNS times 0.15 plus FU times 0.15 plus\nVT times 0.10 plus RI times 0.10]
    end

    R3 --> PM
    R6 --> PM
    SIG --> TS
    SENT --> NS
    PE --> FU
    EG --> FU
    VR --> VT
    BETA --> RI
    VOL --> RI

    PM --> C
    TS --> C
    NS --> C
    FU --> C
    VT --> C
    RI --> C

Key design decisions in the scoring:

• Fundamentals uses inverted P/E — a P/E of 10 scores higher than 50. But P/E alone is misleading, so it is blended 50/50 with earnings growth rate. A low P/E with shrinking earnings is not a bargain.
• Risk is inverse — lower beta and volatility produce a higher risk sub-score. This means a score of 80 for a defensive utility stock means something different than 80 for a speculative small-cap.
• Normalisation bounds are chosen empirically — 3-month returns outside -50%/+50% are clamped, net signal weight outside -8/+8 is clamped. These bounds cover normal market conditions without extreme events dominating.

Part 5: Bearish Alerts and Predictive Score

Bearish Alert Threshold

A stock appears on the Bearish Alerts page when it accumulates 4 or more of these specific signals simultaneously:

The threshold of 4 exists because individual signals are noisy. A single Death Cross without other confirmation has a high false positive rate — markets recover from death crosses regularly. Four concurrent signals firing is a much stronger warning.

Predictive Score

Beyond the current composite score, the pipeline computes a forward-looking prediction using linear regression on each stock’s score history:

flowchart TD
    SH[Score history\nlast 30 trading days] --> LR[Linear regression\nslope and R-squared]
    LR --> SLOPE{Slope direction}
    SLOPE -->|Positive slope\nR2 above 0.3| IMP[Improving\nScore likely rising]
    SLOPE -->|Negative slope\nR2 above 0.3| DEC[Declining\nScore likely falling]
    SLOPE -->|Low R2 or\nnear zero slope| STA[Stable\nNo clear trend]

    IMP --> CONF[Confidence rating\nbased on R-squared]
    DEC --> CONF
    STA --> CONF

    CONF -->|R2 above 0.6| HIGH[High confidence]
    CONF -->|R2 0.3 to 0.6| MED[Medium confidence]
    CONF -->|R2 below 0.3| LOW[Low confidence]

R² (coefficient of determination) measures how well the linear trend fits the data. An R² of 0.7 means 70% of the score variation is explained by the linear trend — a meaningful signal. R² near 0 means the score has been erratic with no clear direction.

This is not a price prediction. It predicts whether the composite score (which aggregates momentum, technicals, and sentiment) is likely to improve or deteriorate — a useful signal for timing entries and exits.

Part 6: The Dashboard

The React PWA is built with Vite and Tailwind, deployed to GitHub Pages. It installs on mobile like a native app and caches data for offline use via a service worker.

Page Structure

flowchart TD
    App[App.tsx] --> OV[Overview\nMarket health\nTop and bottom performers]
    App --> SC[Screener\nAll 220 stocks\nSortable filterable table]
    App --> BA[Bearish Alerts\nStocks with 4 plus signals]
    App --> NF[News and Sentiment\nHeadlines scored by FinBERT]
    App --> SD[Stock Detail\nFull analysis per stock]
    App --> PT[Paper Trading\nSimulated portfolio journal]
    App --> MD[Macro Dashboard\nEconomic indicators]

    SD --> SG[Score Gauge\n0-100 visual]
    SD --> SB[Score Breakdown\n6 factor bars]
    SD --> CC[Candlestick Chart\nOHLCV with volume]
    SD --> VP[Volume Profile\nVPOC value area]
    SD --> SI[Signals list\nbullish and bearish]
    SD --> IN[Insider trades\nSEC EDGAR 90-day]
    SD --> FI[Financials chart\nrevenue earnings margins]
    SD --> NEWS[Recent headlines\nsentiment scored]

Insider Trading Integration

For US stocks, the dashboard shows SEC EDGAR Form 4 insider trading data. This is powerful context — when a CEO buys $500,000 of their own company stock on the open market, it is a strong signal they believe the stock is undervalued.

The ETL:

1. Maps each ticker to its SEC CIK number using EDGAR’s company index
2. Fetches recent Form 4 filings (90-day window)
3. Classifies each transaction: Purchase (P), Sale (S), Award (A), Disposition (D)
4. Calculates net shares bought vs sold in the period
5. Assigns a sentiment: bullish (net buyer), bearish (net seller), neutral

AI Copilot

The dashboard includes a chat assistant that works without any API key using a local rule-based engine:

"What is AAPL score?"         → Score 73/100 breakdown with all 6 factors
"Top 5 stocks this week"      → Ranked list with scores and changes
"Show bearish alerts"         → Filtered list with signal counts
"Compare MSFT and GOOGL"      → Side-by-side score comparison

With an optional LLM API key (Groq free tier supports 14,000 requests/day), it handles open-ended questions using a 70B model with the full stock data as context. Keys are AES-encrypted in localStorage — never sent to any server other than the LLM provider directly.

Part 7: How to Use It

Go to share.devops-monk.com — no account, no subscription.

Decision Framework

flowchart TD
    Start[Open dashboard] --> BA{Check Bearish\nAlert count}
    BA -->|High count\nMarket stress| CAU[Be selective\nRaise quality bar to 70 plus]
    BA -->|Low count\nMarket healthy| NORM[Normal screening\nScore 65 plus is fine]

    CAU --> SCR[Open Screener\nSort by score descending]
    NORM --> SCR

    SCR --> CAND[Find candidate\nScore meets threshold]
    CAND --> CHECK{Check Stock\nDetail page}

    CHECK --> SIG{Any bearish\nsignals active?}
    SIG -->|4 plus signals| AVOID[Avoid - on bearish alert list]
    SIG -->|1 to 3 signals| WATCH[Watch list - wait for improvement]
    SIG -->|0 to 1 signals| SENT2{Sentiment?}

    SENT2 -->|Negative| WAIT[Wait for sentiment to improve]
    SENT2 -->|Neutral or positive| INS{Insider activity?}
    INS -->|Net seller| CAUTION[Proceed with caution]
    INS -->|Neutral or buyer| BUY[Strong candidate]

Score thresholds:

Running Your Own Instance

git clone https://github.com/devops-monk/share-market.git
cd share-market
npm install

# Run the full ETL pipeline (fetches live data, writes to /data)
npm run etl

# Start the dashboard dev server
npm run dev

To deploy your own instance on GitHub Pages:

1. Fork the repository
2. Enable GitHub Pages on the gh-pages branch
3. Add optional secrets for the AI research feature: GROQ_API_KEY, HUGGINGFACE_API_KEY
4. The GitHub Actions workflow runs automatically every hour

The entire stack runs on GitHub’s free tier — zero infrastructure cost for a public repository.

Disclaimer: This dashboard is for educational and informational purposes only. It is not financial advice. Always do your own research and consult a qualified financial advisor before making any investment decisions. Past performance does not guarantee future results.