From Web Data to Stock Options Profits: iTransformer Models
An Exploration of Alternative Data in Options Trading
As the CEO of IcebergData, I want to share our journey of combining web scraping with machine learning to identify options trading opportunities for our .
Our approach isn’t perfect, but we’ve learned valuable lessons about turning public e-commerce data into actionable trading signals working with data collected for our customers.
The Data Foundation: Web Scraping at Scale
We focus on publicly available e-commerce data that most traders overlook. Here are the key metrics we track:
ECOMMERCE_SIGNALS = {
'direct_price_indicators': [
'average_order_value',
'gross_merchandise_value',
'promotion_depth',
'inventory_turnover'
],
'merchant_health': [
'new_seller_registrations',
'seller_ratings',
'inventory_stockouts',
'shipping_delays'
],
'competitive_dynamics': [
'price_matching_frequency',
'category_dominance',
'promotional_calendar',
'market_share_shifts'
]
}Our web scraping infrastructure utilizes a network of distributed proxies and rotating IP addresses to ethically collect publicly available data from e-commerce platforms.
For direct price indicators, we periodically sample product pages to track pricing changes and inventory levels, while aggregating transaction estimates through publicly visible sales counters and stock indicators. Merchant health metrics are gathered from seller profile pages and marketplace statistics, focusing on publicly posted ratings and service metrics. F
or competitive dynamics, we analyze pricing patterns across multiple sellers and categories, tracking how often prices change and match competitors, while monitoring promotional announcements and category rankings that are publicly displayed.
All data collection respects website terms of service and rate limits, with built-in delays between requests to minimize server impact.
We aggregate this information across multiple marketplaces to build a comprehensive view while maintaining data quality through automated validation checks and anomaly detection systems. This allows us to construct reliable indicators without relying on any privileged or private information.
Our web scraping infrastructure collects this data hourly:
class EcommerceDataCollector:
def __init__(self, target_sites):
self.scrapers = self._initialize_scrapers(target_sites)
self.rate_limiter = RateLimiter(max_requests=100)
async def collect_metrics(self):
"""Collect e-commerce metrics while respecting site policies"""
metrics = {}
async with self.rate_limiter:
for site in self.scrapers:
try:
raw_data = await self.scrapers[site].fetch_data()
metrics[site] = self.process_raw_data(raw_data)
except Exception as e:
self.log_error(f"Failed to collect data from {site}: {e}")
return metrics
def process_raw_data(self, raw_data):
"""Extract relevant metrics from scraped HTML"""
return {
'price_data': self._extract_prices(raw_data),
'inventory': self._extract_inventory(raw_data),
'merchant_metrics': self._extract_merchant_data(raw_data)
}The iTransformer Edge
The real breakthrough came when we started using the iTransformer architecture. Unlike traditional models, iTransformer helps us understand how different e-commerce metrics relate to each other:
The key innovation of iTransformer lies in how it processes multivariate data. Traditional transformers treat time as the primary dimension, looking at all variables at each timestamp. Instead, iTransformer inverts this approach by treating each variable (like inventory levels, price changes, or seller metrics) as a primary token sequence across time. This inversion allows the attention mechanism to directly model relationships between different e-commerce metrics, rather than just their temporal patterns.
For example,
when we feed in scraped data, the model can identify how changes in inventory levels correlate with pricing strategies across different sellers, or how promotional events affect overall market dynamics.
By learning these cross-metric relationships, iTransformer can better predict which signals might lead to significant stock options movements.
We’ve found this particularly valuable for identifying emerging trends before they become apparent in traditional market indicators, giving us a crucial edge in options trading where timing and direction are everything.
class MarketSignalTransformer(nn.Module):
def __init__(self, n_features, d_model):
super().__init__()
self.feature_embedding = nn.Linear(n_features, d_model)
self.iTransformer = iTransformerBlock(d_model)
self.options_head = OptionsPredictor(d_model)
def forward(self, x):
# x shape: (batch, features, timesteps)
embedded = self.feature_embedding(x)
# Key difference: focus on relationships between features
transformed = self.iTransformer(embedded)
# Predict options movements
return self.options_head(transformed)Connecting Data to Options Trading
The relationship between web-scraped e-commerce metrics and options contracts is particularly compelling due to options’ sensitivity to volatility and market expectations.
Why options instead of stock ?
While stock prices primarily reflect current value, options prices are heavily influenced by anticipated future movements and their magnitude.
Our scraped metrics — like rapid inventory changes, pricing pattern shifts, and seller behavior — often signal upcoming periods of increased volatility or directional momentum before they impact stock prices.
For instance, when our system detects unusual patterns in inventory turnover or aggressive price matching across multiple sellers, it indicates a surge in market activity. These signals are especially valuable for options trading because they help predict not just direction but also the timing and intensity of potential moves.
The advantage of using options :
Options contracts, with inherent leverage and time decay, are ideal instruments for capitalizing on such temporary market inefficiencies.
Additionally, the rich granularity of our e-commerce data helps us gauge market sentiment and potential reaction magnitude, which directly relates to options’ implied volatility — a crucial component of options pricing that isn’t as relevant for straight stock trading.
How do we map our scraped data to options opportunities?
class OptionsSignalGenerator:
def __init__(self, model, threshold=0.75):
self.model = model
self.threshold = threshold
def analyze_trading_opportunity(self, scraped_data):
"""Convert web-scraped data into options trading signals"""
# Normalize incoming data
processed_data = self.preprocess_scraped_data(scraped_data)
# Generate predictions using iTransformer
prediction = self.model(processed_data)
if prediction.confidence > self.threshold:
return self.generate_options_strategy(prediction)
return None
def generate_options_strategy(self, prediction):
return {
'contract_type': 'CALL' if prediction.direction > 0 else 'PUT',
'strike_price': self.calculate_optimal_strike(prediction),
'expiration': self.select_expiration(prediction),
'position_size': self.calculate_position_size(prediction),
'confidence': prediction.confidence
}Major E-commerce Event , BLACK FRIDAY
Let’s look at a recent case where our system identified an opportunity:
event_data = {
'avg_order_value': 127.50,
'inventory_turnover': 3.2, # 3.2x normal rate
'price_matching_instances': 245, # Unusual spike
'merchant_ratings_avg': 4.2
}
# Strategy generated
strategy = {
'thesis': 'Increased inventory turnover suggesting strong sales',
'position': 'OTM Calls',
'expiration': '45 DTE', # Days till expiration
'risk_management': {
'stop_loss': '25%',
'profit_target': '100%',
'position_size': '2% of portfolio'
}
}The web-scraped data revealed a compelling set of market conditions.
Our metrics showed an average order value of $127.50 +10% vs prior 4 weeks, combined with a remarkable inventory turnover rate at 3.2 times the normal pace.
This accelerated turnover rate was particularly noteworthy, suggesting unusually strong consumer demand.
Adding to this picture, we detected 245 instances of price matching among sellers, representing a significant spike in competitive pricing activity.
Despite this intense market activity, merchant ratings remained strong at 4.2 out of 5, indicating that sellers were managing the high-volume period effectively without compromising service quality.
These signals led us to develop a focused options trading strategy.
We opted for Out-of-The-Money (OTM) 45 days calls, reasoning that the exceptional inventory turnover rate strongly suggested potential revenue outperformance.
The 45-day expiration window was strategically chosen to capture the period when these strong sales metrics would likely translate into tangible financial results including the next earnings announcement. This timeframe provided sufficient duration for market recognition of the strong sales trend while managing the impact of time decay on our options positions.
Our risk management framework was designed to be both protective and profit-oriented. We implemented a 25% stop-loss threshold to contain potential downsides if our analysis proved incorrect, while setting a 100% profit target to capitalize on our conviction in the signals.
Position sizing was kept to 2% of the portfolio as our trading policy restrict us for risk management.
Current Results and Ongoing Challenges
Our approach has shown promise, but we remain humble about its limitations:
- 65% win rate on options trades (after accounting for fees)
- Average return of 31% per successful trade
- But also: 15% average loss on unsuccessful trades
Key challenges we’re still working on:
- Data quality and consistency
- Market noise vs. actual signals
- Options pricing inefficiencies
Future Improvements
We’re continuously working to enhance our system:
- Expanding our data collection to market in australia and asia.
- Refining our iTransformer architecture by implementing new attention methodologies.
- Improving risk management for options positions by considering Fx Exposure.
The Right Way to Trading Systems
A few important notes about our approach:
- We only trade highly liquid options
- Position sizing is critical
- We never bet on binary events
- Risk management always comes first
def risk_management_example():
def validate_trade(signal, market_data):
checks = {
'liquidity': check_option_liquidity(market_data),
'spread': check_bid_ask_spread(market_data),
'position_size': calculate_safe_position(signal),
'overall_exposure': check_portfolio_exposure()
}
return all(checks.values())Conclusion
Web scraping combined with iTransformer models has opened new opportunities in options trading. While we’re excited about our progress, we remain focused on continuous improvement and responsible trading practices.
Our edge comes from three key elements:
- Quality web-scraped data
- Innovative use of iTransformer architecture
- Disciplined options trading strategy
We believe this is about consistent edges through better data and more realistic analysis.
Disclaimer: Trading options involves substantial risk of loss. Our experience may not be representative of future results. Always conduct your own research and consider your risk tolerance before trading
Note: Some details have been modified for confidentiality. The code provided is for educational purposes and should not be used for actual trading without proper risk management and validation.
If you found this article helpful, please follow me for more content on machine learning and quantitative finance.
