feat: x services (#43)
* feat: twitter services * fix: package.json * fix: type error * refactor: launcher * refactor: BrowserBase/Stagehand * fix: config & startup * chore: i18n * fix: continue i18n * feat: cookie login * fix: save cookie to session json * docs: architecture * chore: remove cli & use mcp adapter for mcp-server * refactor: remove browser adapter, use playwright directly * refactor: remove launcher services, replace twitter.com to x.com * feat: load session to auto login * docs: update architecture md * chore: more debug logs * feat: tweet parser without hast and rehype * fix: type error * fix: env example
This commit is contained in:
RainbowBird
@@ -35,3 +35,5 @@ coverage/
|
||||
*.mp3
|
||||
|
||||
**/temp/
|
||||
|
||||
twitter-session.json
|
||||
|
||||
@@ -15,6 +15,7 @@ words:
|
||||
- baiducloud
|
||||
- bigserial
|
||||
- Bitstream
|
||||
- browserbasehq
|
||||
- bumpp
|
||||
- catppuccin
|
||||
- changelogithub
|
||||
|
||||
Generated
+734
-116
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,21 @@
|
||||
# Browser Config
|
||||
BROWSER_HEADLESS=false
|
||||
BROWSER_USER_AGENT=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36
|
||||
BROWSER_VIEWPORT_WIDTH=1280
|
||||
BROWSER_VIEWPORT_HEIGHT=800
|
||||
BROWSER_TIMEOUT=30000
|
||||
BROWSER_REQUEST_TIMEOUT=20000
|
||||
BROWSER_REQUEST_RETRIES=2
|
||||
|
||||
# Adapter Config
|
||||
ENABLE_AIRI=false
|
||||
AIRI_URL=http://localhost:3000
|
||||
AIRI_TOKEN=your_airi_token
|
||||
|
||||
ENABLE_MCP=true
|
||||
MCP_PORT=8080
|
||||
|
||||
# System Config
|
||||
LOG_LEVEL=info # Optional: error, warn, info, verbose, debug
|
||||
LOG_FORMAT=pretty # Optional: json, pretty
|
||||
CONCURRENCY=1
|
||||
@@ -0,0 +1,396 @@
|
||||
# Twitter Service Architecture Documentation
|
||||
|
||||
## 1. Project Overview
|
||||
|
||||
Twitter Service is a web automation service based on BrowserBase, providing structured access and interaction capabilities with Twitter data. It employs a layered architecture design that supports multiple adapters for integration with different applications.
|
||||
|
||||
## 2. Design Goals
|
||||
|
||||
- **Reliability**: Stable handling of Twitter page changes and limitations
|
||||
- **Scalability**: Easy to add new features and support different integration methods
|
||||
- **Performance Optimization**: Intelligent management of request frequency and browser sessions
|
||||
- **Data Structuring**: Provides standardized, typed data models
|
||||
|
||||
## 3. Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Application/Consumer Layer │
|
||||
│ │
|
||||
│ ┌────────────┐ ┌─────────────┐ │
|
||||
│ │ │ │ │ │
|
||||
│ │ Airi Core │ │ Other LLM │ │
|
||||
│ │ │ │ Applications│ │
|
||||
│ │ │ │ │ │
|
||||
│ └──────┬─────┘ └──────┬──────┘ │
|
||||
└──────────┼─────────────────────┼────────────┘
|
||||
│ │
|
||||
┌──────────▼─────────────────────▼────────────┐
|
||||
│ Adapter Layer │
|
||||
│ │
|
||||
│ ┌────────────┐ ┌─────────────┐ │
|
||||
│ │Airi Adapter│ │ MCP Adapter │ │
|
||||
│ │(@server-sdk)│ │ (HTTP/JSON) │ │
|
||||
│ └──────┬─────┘ └──────┬──────┘ │
|
||||
└──────────┼─────────────────────┼────────────┘
|
||||
│ │
|
||||
┌──────────▼─────────────────────▼────────────┐
|
||||
│ Core Services Layer │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────┐ │
|
||||
│ │ Twitter Services │ │
|
||||
│ │ │ │
|
||||
│ │ ┌────────┐ ┌────────────┐ │ │
|
||||
│ │ │ Auth │ │ Timeline │ │ │
|
||||
│ │ │ Service│ │ Service │ │ │
|
||||
│ │ └────────┘ └────────────┘ │ │
|
||||
│ │ │ │
|
||||
│ └──────────────────┬───────────────┘ │
|
||||
└──────────────────────┼──────────────────────┘
|
||||
│
|
||||
┌───────────▼────────────┐
|
||||
│ Browser Adapter Layer │
|
||||
│ (BrowserAdapter) │
|
||||
└───────────┬────────────┘
|
||||
│
|
||||
┌───────────▼────────────┐
|
||||
│ Stagehand │
|
||||
└───────────┬────────────┘
|
||||
│
|
||||
┌───────────▼────────────┐
|
||||
│ Playwright │
|
||||
└────────────────────────┘
|
||||
```
|
||||
|
||||
## 4. Technology Stack and Dependencies
|
||||
|
||||
- **Core Library**: TypeScript, Node.js
|
||||
- **Browser Automation**: BrowserBase Stagehand, Playwright
|
||||
- **HTML Parsing**: unified, rehype-parse, unist-util-visit
|
||||
- **API Server**: H3.js, listhen
|
||||
- **Adapters**: Airi Server SDK, MCP SDK
|
||||
- **Logging System**: @guiiai/logg
|
||||
- **Configuration**: defu (deep merging configurations)
|
||||
- **Utility Library**: zod (type validation)
|
||||
|
||||
## 5. Key Components
|
||||
|
||||
### 5.1 Adapter Layer
|
||||
|
||||
#### 5.1.1 Airi Adapter
|
||||
|
||||
Provides integration with the Airi LLM platform, handling event-driven communication.
|
||||
|
||||
#### 5.1.2 MCP Adapter
|
||||
|
||||
Implements the Model Context Protocol interface, providing communication based on HTTP. Currently using the official MCP SDK implementation, providing high-performance HTTP server and SSE communication through H3.js.
|
||||
|
||||
The MCP adapter exposes several tools and resources:
|
||||
|
||||
- **Timeline Resource**: Access tweets from the user's timeline
|
||||
- **Tweet Details Resource**: Get detailed information about a specific tweet
|
||||
- **User Profile Resource**: Retrieve user profile information
|
||||
|
||||
Additionally, it provides tools for interaction:
|
||||
|
||||
- **Login Tool**: Simplified authentication tool that provides clear feedback on session status. It attempts to load existing sessions, and clearly communicates whether a session was loaded successfully or if manual login is required. The tool no longer requires username/password parameters, as it relies on the enhanced session management system.
|
||||
- **Post Tweet Tool**: Create and publish new tweets
|
||||
- **Like Tweet Tool**: Like a tweet by its ID
|
||||
- **Retweet Tool**: Retweet a tweet by its ID
|
||||
- **Refresh Timeline Tool**: Refresh the timeline with the latest tweets, with options to control the count and whether to include replies and retweets.
|
||||
- **Get My Profile Tool**: Get information about a user's profile. It can extract the username from the current URL or accept a specific username as a parameter.
|
||||
|
||||
The adapter uses internationalized messages (Chinese/English) to provide clear feedback to users about login status and session management.
|
||||
|
||||
#### 5.1.3 Development Server
|
||||
|
||||
Using listhen for optimized development experience, including automatic browser opening, real-time logging, and debugging tools.
|
||||
|
||||
### 5.2 Core Service Layer
|
||||
|
||||
#### 5.2.1 Authentication Service (Auth Service)
|
||||
|
||||
The Authentication Service has been significantly enhanced to improve reliability and error handling:
|
||||
|
||||
1. **Improved Session Detection**: Enhanced logic for detecting existing browser sessions
|
||||
2. **Robust Error Handling**: Implemented granular error handling to distinguish between different authentication failure types
|
||||
3. **Timeout Optimization**: Adjusted timeouts for various operations to enhance stability during network fluctuations
|
||||
4. **Enhanced Cookie Management**: Improved cookie storage and loading mechanisms to reduce the need for manual login
|
||||
5. **Session Validation**: Added comprehensive session validation to verify the integrity of saved sessions
|
||||
6. **Simplified API**: Removed the need for explicit username/password in the login method, relying instead on session files and browser session detection
|
||||
|
||||
The service follows a multi-stage authentication approach:
|
||||
|
||||
1. **Session File Loading**: First attempts to load saved sessions from disk
|
||||
2. **Existing Session Detection**: Checks if the browser already has a valid Twitter session
|
||||
3. **Manual Login Process**: If necessary, guides through the Twitter login page
|
||||
|
||||
After successful authentication through any method, sessions are automatically persisted for future use. The system provides clear feedback to users about the current login state and automatically monitors and saves sessions when changes are detected.
|
||||
|
||||
#### 5.2.2 Timeline Service (Timeline Service)
|
||||
|
||||
Gets and processes Twitter timeline content.
|
||||
|
||||
#### 5.2.3 Other Services
|
||||
|
||||
Includes search service, interaction service, user profile service, etc. (not implemented in MVP)
|
||||
|
||||
### 5.3 Parsers and Tools
|
||||
|
||||
#### 5.3.1 Tweet Parser
|
||||
|
||||
Extracts structured data from HTML.
|
||||
|
||||
#### 5.3.2 Rate Limiter
|
||||
|
||||
Controls request frequency to avoid triggering Twitter limits.
|
||||
|
||||
#### 5.3.3 Session Manager
|
||||
|
||||
Manages authentication session data, providing methods to:
|
||||
|
||||
- Save session cookies to local files
|
||||
- Load previous sessions during startup
|
||||
- Delete invalid or expired sessions
|
||||
- Validate session age and integrity
|
||||
|
||||
### 5.3.4 Browser Adapter Layer
|
||||
|
||||
The service has migrated from direct BrowserBase API usage to Stagehand, an AI-powered web browsing framework built on top of Playwright. Stagehand offers three core APIs that simplify browser automation:
|
||||
|
||||
- **act**: Execute actions on the page through natural language instructions
|
||||
- **extract**: Retrieve structured data from the page using natural language queries
|
||||
- **observe**: Analyze the page and suggest possible actions before execution
|
||||
|
||||
Stagehand processes the DOM in chunks to optimize LLM performance and provides fallback vision capabilities for complex page structures. This migration significantly improves code maintainability and automation reliability when interacting with Twitter's interface.
|
||||
|
||||
## 6. Data Flow
|
||||
|
||||
1. **Request Flow**: Application Layer → Adapter → Core Service → Browser Adapter Layer → BrowserBase API → Twitter
|
||||
2. **Response Flow**: Twitter → BrowserBase API → Browser Adapter Layer → Core Service → Data Parsing → Adapter → Application Layer
|
||||
3. **Authentication Flow**:
|
||||
- Load Session → Check Existing Session → Manual Login → Session Validation → Session Storage
|
||||
- Clear feedback is provided at each step of the authentication process
|
||||
|
||||
## 7. Configuration System
|
||||
|
||||
The configuration system has been optimized using the `defu` library for deep merging configurations, eliminating redundant initialization. The updated configuration structure includes Stagehand-specific settings:
|
||||
|
||||
```typescript
|
||||
interface Config {
|
||||
// BrowserBase/Stagehand configuration
|
||||
browserbase: {
|
||||
apiKey: string
|
||||
projectId?: string
|
||||
endpoint?: string
|
||||
stagehand?: {
|
||||
modelName?: string // e.g., "gpt-4o" or "claude-3-5-sonnet-latest"
|
||||
modelClientOptions?: {
|
||||
apiKey: string // OpenAI or Anthropic API key
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Browser configuration
|
||||
browser: BrowserConfig
|
||||
|
||||
// Twitter configuration
|
||||
twitter: {
|
||||
credentials?: TwitterCredentials
|
||||
defaultOptions?: {
|
||||
timeline?: TimelineOptions
|
||||
search?: SearchOptions
|
||||
}
|
||||
}
|
||||
|
||||
// Adapter configuration
|
||||
adapters: {
|
||||
airi?: {
|
||||
url?: string
|
||||
token?: string
|
||||
enabled: boolean
|
||||
}
|
||||
mcp?: {
|
||||
port?: number
|
||||
enabled: boolean
|
||||
}
|
||||
}
|
||||
|
||||
// System configuration
|
||||
system: {
|
||||
logLevel: string
|
||||
concurrency: number
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The system no longer relies on the `TWITTER_COOKIES` environment variable, as cookies are now managed through the session management system.
|
||||
|
||||
## 8. Development and Testing
|
||||
|
||||
### 8.1 Development Environment Setup
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Set environment variables
|
||||
cp .env.example .env
|
||||
# Edit .env to add BrowserBase API key and Twitter credentials (optional)
|
||||
|
||||
# Development mode startup
|
||||
npm run dev # Standard mode
|
||||
npm run dev:mcp # MCP development server mode
|
||||
```
|
||||
|
||||
### 8.2 Testing Strategy
|
||||
|
||||
- **Unit Tests**: Test parsers, utility classes, and business logic
|
||||
- **Integration Tests**: Test service and adapter interaction
|
||||
- **End-to-End Tests**: Simulate complete usage scenarios
|
||||
|
||||
## 9. Integration Example
|
||||
|
||||
### 9.1 Integration Example with Stagehand
|
||||
|
||||
```typescript
|
||||
import { StagehandAdapter, TwitterService } from 'twitter-services'
|
||||
|
||||
async function main() {
|
||||
// Initialize Stagehand adapter
|
||||
const browser = new StagehandAdapter(process.env.BROWSERBASE_API_KEY, process.env.BROWSERBASE_PROJECT_ID)
|
||||
await browser.initialize({
|
||||
headless: true,
|
||||
stagehand: {
|
||||
modelName: 'gpt-4o', // Or 'claude-3-5-sonnet-latest' for Anthropic
|
||||
modelClientOptions: {
|
||||
apiKey: process.env.OPENAI_API_KEY // Or process.env.ANTHROPIC_API_KEY
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
// Create Twitter service
|
||||
const twitter = new TwitterService(browser)
|
||||
|
||||
// Authenticate - will try multi-stage approach
|
||||
const loggedIn = await twitter.login()
|
||||
|
||||
if (loggedIn) {
|
||||
console.log('Login successful')
|
||||
|
||||
// Get timeline using natural language capabilities of Stagehand
|
||||
const tweets = await twitter.getTimeline({ count: 10 })
|
||||
console.log(tweets)
|
||||
}
|
||||
else {
|
||||
console.error('Login failed')
|
||||
}
|
||||
|
||||
// Release resources
|
||||
await browser.close()
|
||||
}
|
||||
```
|
||||
|
||||
### 9.2 Integrating as Airi Module
|
||||
|
||||
```typescript
|
||||
import { AiriAdapter, BrowserBaseMCPAdapter, TwitterService } from 'twitter-services'
|
||||
|
||||
async function startAiriModule() {
|
||||
const browser = new BrowserBaseMCPAdapter(process.env.BROWSERBASE_API_KEY)
|
||||
await browser.initialize({ headless: true })
|
||||
|
||||
const twitter = new TwitterService(browser)
|
||||
|
||||
// Create Airi adapter
|
||||
const airiAdapter = new AiriAdapter(twitter, {
|
||||
url: process.env.AIRI_URL,
|
||||
token: process.env.AIRI_TOKEN
|
||||
})
|
||||
|
||||
// Start adapter
|
||||
await airiAdapter.start()
|
||||
|
||||
console.log('Twitter service running as Airi module')
|
||||
}
|
||||
```
|
||||
|
||||
### 9.3 Using MCP for Integration
|
||||
|
||||
```typescript
|
||||
// Use MCP SDK to interact with Twitter service
|
||||
import { Client } from '@modelcontextprotocol/sdk/client/index.js'
|
||||
import { SSEClientTransport } from '@modelcontextprotocol/sdk/client/sse.js'
|
||||
|
||||
async function connectToTwitterService() {
|
||||
// Create SSE transport
|
||||
const transport = new SSEClientTransport('http://localhost:8080/sse', 'http://localhost:8080/messages')
|
||||
|
||||
// Create client
|
||||
const client = new Client()
|
||||
await client.connect(transport)
|
||||
|
||||
// Get timeline
|
||||
const timeline = await client.get('twitter://timeline/10')
|
||||
console.log('Timeline:', timeline.contents)
|
||||
|
||||
// Use simplified login tool without parameters
|
||||
const loginResult = await client.useTool('login', {})
|
||||
console.log('Login result:', loginResult.content[0].text)
|
||||
|
||||
// Use refresh timeline tool to get latest tweets
|
||||
const refreshResult = await client.useTool('refresh-timeline', { count: 15, includeReplies: false })
|
||||
console.log('Refresh result:', refreshResult.content[0].text)
|
||||
console.log('New tweets:', refreshResult.resources)
|
||||
|
||||
// Get user profile information
|
||||
const profileResult = await client.useTool('get-my-profile', { username: 'twitter' })
|
||||
console.log('Profile info:', profileResult.content[0].text)
|
||||
|
||||
// Use tool to send tweet
|
||||
const result = await client.useTool('post-tweet', { content: 'Hello from MCP!' })
|
||||
console.log('Result:', result.content)
|
||||
|
||||
return client
|
||||
}
|
||||
```
|
||||
|
||||
## 10. Extension Guide
|
||||
|
||||
### 10.1 Adding New Features
|
||||
|
||||
For example, adding "Get Tweets from a Specific User" functionality:
|
||||
|
||||
1. Extend the interface in `src/types/twitter.ts`
|
||||
2. Implement the method in `src/core/twitter-service.ts`
|
||||
3. Add corresponding handling logic in the adapter
|
||||
4. If it's an MCP adapter, add appropriate resources or tools in `configureServer()`
|
||||
|
||||
### 10.2 Supporting New Adapters
|
||||
|
||||
1. Create a new adapter class
|
||||
2. Implement communication logic with the target system
|
||||
3. Add configuration support in the entry file
|
||||
|
||||
## 11. Maintenance Recommendations
|
||||
|
||||
- **Automated Testing**: Write unit tests and integration tests
|
||||
- **Monitoring & Alerts**: Monitor service status and Twitter access limitations
|
||||
- **Selector Updates**: Regularly validate and update selector configurations
|
||||
- **Session Management**: Use the built-in session management system to improve stability and reduce manual login requirements. Consider implementing session rotation and validation.
|
||||
- **Cookie Management**: The system now automatically manages cookie storage via the SessionManager, but consider adding encrypted storage for production environments.
|
||||
- **User Feedback**: Maintain clear, internationalized feedback messages for authentication status to improve user experience.
|
||||
|
||||
### 11.4 Stagehand Maintenance
|
||||
|
||||
- **Model Selection**: Regularly evaluate the performance of different LLM models (GPT-4o, Claude 3.5 Sonnet) for your specific use cases
|
||||
- **Prompt Engineering**: Refine natural language instructions to improve reliability and performance
|
||||
- **Vision Capabilities**: Consider enabling vision capabilities for complex DOM structures by setting `useVision: true` in appropriate operations
|
||||
- **DOM Chunking**: Monitor and optimize chunk sizes based on the complexity of the Twitter interface
|
||||
|
||||
## 12. Project Roadmap
|
||||
|
||||
- MVP Stage: Core functionality with Stagehand integration (authentication, browsing timeline)
|
||||
- Stage Two: Enhanced interaction features utilizing Stagehand's natural language capabilities
|
||||
- Stage Three: Advanced search and filtering features with optimized LLM prompts
|
||||
- Stage Four: Performance optimization and multi-model support
|
||||
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"name": "@proj-airi/twitter-services",
|
||||
"type": "module",
|
||||
"version": "0.1.0",
|
||||
"description": "Twitter Services for MCP",
|
||||
"author": "RainbowBird <rbxin2003@outlook.com>",
|
||||
"license": "MIT",
|
||||
"scripts": {
|
||||
"dev": "tsx src/main.ts",
|
||||
"mcp:ui": "pnpx @modelcontextprotocol/inspector",
|
||||
"postinstall": "playwright install chromium"
|
||||
},
|
||||
"dependencies": {
|
||||
"@browserbasehq/stagehand": "^1.13.1",
|
||||
"@guiiai/logg": "^1.0.0",
|
||||
"@modelcontextprotocol/sdk": "^1.6.1",
|
||||
"@proj-airi/server-sdk": "^0.1.0",
|
||||
"defu": "^6.1.4",
|
||||
"dotenv": "^16.4.7",
|
||||
"h3": "^1.11.0",
|
||||
"listhen": "^1.6.0",
|
||||
"ofetch": "^1.3.3",
|
||||
"playwright": "^1.50.1",
|
||||
"zod": "^3.24.2"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^18.16.3",
|
||||
"tsx": "^4.19.0",
|
||||
"typescript": "^5.0.4"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
/**
|
||||
* Airi Adapter
|
||||
* Adapts the Twitter service as an Airi module
|
||||
*/
|
||||
export class AiriAdapter {}
|
||||
@@ -0,0 +1,577 @@
|
||||
import type { TwitterService } from '../core/twitter-service'
|
||||
|
||||
import { Buffer } from 'node:buffer'
|
||||
import { createServer } from 'node:http'
|
||||
import { McpServer, ResourceTemplate } from '@modelcontextprotocol/sdk/server/mcp.js'
|
||||
import { SSEServerTransport } from '@modelcontextprotocol/sdk/server/sse.js'
|
||||
import { createApp, createRouter, defineEventHandler, toNodeListener } from 'h3'
|
||||
import { z } from 'zod'
|
||||
|
||||
import { errorToMessage } from '../utils/error'
|
||||
import { logger } from '../utils/logger'
|
||||
|
||||
/**
|
||||
* MCP Protocol Adapter
|
||||
* Adapts the Twitter service to MCP protocol using official MCP SDK
|
||||
* Implements HTTP server using H3.js
|
||||
*/
|
||||
export class MCPAdapter {
|
||||
private twitterService: TwitterService
|
||||
private mcpServer: McpServer
|
||||
private app: ReturnType<typeof createApp>
|
||||
private server: ReturnType<typeof createServer> | null = null
|
||||
private port: number
|
||||
private activeTransports: SSEServerTransport[] = []
|
||||
private extraResourceInfo: string[] = []
|
||||
|
||||
constructor(twitterService: TwitterService, port: number = 8080) {
|
||||
this.twitterService = twitterService
|
||||
this.port = port
|
||||
|
||||
// Create MCP server
|
||||
this.mcpServer = new McpServer({
|
||||
name: 'Twitter Service',
|
||||
version: '1.0.0',
|
||||
})
|
||||
|
||||
// Create H3 app
|
||||
this.app = createApp()
|
||||
|
||||
// Configure resources and tools
|
||||
this.configureServer()
|
||||
|
||||
// Set up H3 routes
|
||||
this.setupRoutes()
|
||||
}
|
||||
|
||||
/**
|
||||
* Configure MCP server resources and tools
|
||||
*/
|
||||
private configureServer(): void {
|
||||
logger.mcp.debug('Configuring MCP server resources and tools...')
|
||||
|
||||
// Add timeline resource with improved registration
|
||||
this.mcpServer.resource(
|
||||
'timeline',
|
||||
new ResourceTemplate('twitter://timeline/{count}', {
|
||||
list: async () => {
|
||||
logger.mcp.debug('Listing available timeline resources')
|
||||
return {
|
||||
resources: [{
|
||||
name: 'timeline',
|
||||
uri: 'twitter://timeline/10', // Default number of tweets
|
||||
description: 'Tweet timeline',
|
||||
}],
|
||||
}
|
||||
},
|
||||
}),
|
||||
async (uri: URL, { count }: { count?: string }) => {
|
||||
try {
|
||||
logger.mcp.withField('uri', uri.toString()).withField('count', count || 'default').debug('Getting timeline')
|
||||
|
||||
const tweets = await this.twitterService.getTimeline({
|
||||
count: count ? Number.parseInt(count) : undefined,
|
||||
})
|
||||
|
||||
logger.mcp.withField('tweetCount', tweets.length).debug('Successfully retrieved timeline tweets')
|
||||
|
||||
return {
|
||||
contents: tweets.map(tweet => ({
|
||||
uri: `twitter://tweet/${tweet.id}`,
|
||||
text: `Tweet by @${tweet.author.username} (${tweet.author.displayName}):\n${tweet.text}`,
|
||||
})),
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
logger.mcp.errorWithError('Failed to get timeline:', error)
|
||||
return { contents: [] }
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
// Add tweet details resource
|
||||
this.mcpServer.resource(
|
||||
'tweet',
|
||||
new ResourceTemplate('twitter://tweet/{id}', { list: undefined }),
|
||||
async (uri: URL, { id }) => {
|
||||
try {
|
||||
const tweet = await this.twitterService.getTweetDetails(id as string)
|
||||
|
||||
return {
|
||||
contents: [{
|
||||
uri: uri.href,
|
||||
text: `Tweet by @${tweet.author.username} (${tweet.author.displayName}):\n${tweet.text}`,
|
||||
}],
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
logger.mcp.errorWithError('Error fetching tweet details:', error)
|
||||
return { contents: [] }
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
// Add user profile resource
|
||||
this.mcpServer.resource(
|
||||
'profile',
|
||||
new ResourceTemplate('twitter://user/{username}', { list: undefined }),
|
||||
async (uri, { username }) => {
|
||||
try {
|
||||
const profile = await this.twitterService.getUserProfile(username as string)
|
||||
|
||||
return {
|
||||
contents: [{
|
||||
uri: uri.href,
|
||||
text: `Profile for @${profile.username} (${profile.displayName})\n${profile.bio || ''}`,
|
||||
}],
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
logger.mcp.errorWithError('Error fetching user profile:', error)
|
||||
return { contents: [] }
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
// Add login tool
|
||||
this.mcpServer.tool(
|
||||
'login',
|
||||
{},
|
||||
async () => {
|
||||
try {
|
||||
const success = await this.twitterService.login()
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: success
|
||||
? 'Successfully loaded login state from session file! If you logged in manually, auto-monitoring is set up to save your session.'
|
||||
: 'No valid session file found. Please log in manually in the browser, the system will automatically save your session.',
|
||||
}],
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Failed to check login status: ${errorToMessage(error)}` }],
|
||||
isError: true,
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
// Add post tweet tool
|
||||
this.mcpServer.tool(
|
||||
'post-tweet',
|
||||
{
|
||||
content: z.string(),
|
||||
replyTo: z.string().optional(),
|
||||
media: z.array(z.string()).optional(),
|
||||
},
|
||||
async ({ content, replyTo, media }) => {
|
||||
try {
|
||||
const tweetId = await this.twitterService.postTweet(content, {
|
||||
inReplyTo: replyTo,
|
||||
media,
|
||||
})
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: `Successfully posted tweet: ${tweetId}`,
|
||||
}],
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Failed to post tweet: ${errorToMessage(error)}` }],
|
||||
isError: true,
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
// Add like tweet tool
|
||||
this.mcpServer.tool(
|
||||
'like-tweet',
|
||||
{ tweetId: z.string() },
|
||||
async ({ tweetId }) => {
|
||||
try {
|
||||
const success = await this.twitterService.likeTweet(tweetId)
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: success ? 'Successfully liked tweet' : 'Failed to like tweet',
|
||||
}],
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Failed to like tweet: ${errorToMessage(error)}` }],
|
||||
isError: true,
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
// Add retweet tool
|
||||
this.mcpServer.tool(
|
||||
'retweet',
|
||||
{ tweetId: z.string() },
|
||||
async ({ tweetId }) => {
|
||||
try {
|
||||
const success = await this.twitterService.retweet(tweetId)
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: success ? 'Successfully retweeted' : 'Failed to retweet',
|
||||
}],
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Failed to retweet: ${errorToMessage(error)}` }],
|
||||
isError: true,
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
// Add save session tool
|
||||
this.mcpServer.tool(
|
||||
'save-session',
|
||||
{},
|
||||
async () => {
|
||||
try {
|
||||
const success = await this.twitterService.saveSession()
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: success
|
||||
? 'Successfully saved browser session to file. This session will be loaded automatically next time.'
|
||||
: 'Failed to save browser session',
|
||||
}],
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Failed to save session: ${errorToMessage(error)}` }],
|
||||
isError: true,
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
// Add search tool
|
||||
this.mcpServer.tool(
|
||||
'search',
|
||||
{
|
||||
query: z.string(),
|
||||
count: z.number().optional(),
|
||||
filter: z.enum(['latest', 'photos', 'videos', 'top']).optional(),
|
||||
},
|
||||
async ({ query, count, filter }) => {
|
||||
try {
|
||||
const results = await this.twitterService.searchTweets(query, { count, filter })
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: `Search results: ${results.length} tweets`,
|
||||
}],
|
||||
resources: results.map(tweet => `twitter://tweet/${tweet.id}`),
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Search failed: ${errorToMessage(error)}` }],
|
||||
isError: true,
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
// Add refresh timeline tool
|
||||
this.mcpServer.tool(
|
||||
'refresh-timeline',
|
||||
{
|
||||
count: z.number().optional(),
|
||||
includeReplies: z.boolean().optional(),
|
||||
includeRetweets: z.boolean().optional(),
|
||||
},
|
||||
async ({ count, includeReplies, includeRetweets }) => {
|
||||
try {
|
||||
const tweets = await this.twitterService.getTimeline({
|
||||
count,
|
||||
includeReplies,
|
||||
includeRetweets,
|
||||
})
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: `Successfully refreshed timeline, retrieved ${tweets.length} tweets`,
|
||||
}],
|
||||
resources: tweets.map(tweet => `twitter://tweet/${tweet.id}`),
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Failed to refresh timeline: ${errorToMessage(error)}` }],
|
||||
isError: true,
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
// Add get my profile tool
|
||||
this.mcpServer.tool(
|
||||
'get-my-profile',
|
||||
{
|
||||
username: z.string().optional(),
|
||||
},
|
||||
async ({ username }) => {
|
||||
try {
|
||||
let profileUsername = username
|
||||
|
||||
// If no username provided, try to get from current URL
|
||||
if (!profileUsername) {
|
||||
const currentUrl = await this.twitterService.getCurrentUrl()
|
||||
profileUsername = this.extractUsernameFromUrl(currentUrl)
|
||||
}
|
||||
|
||||
// If we still don't have a username, return an error
|
||||
if (!profileUsername) {
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: `Failed to get profile: Please provide a username or navigate to a profile page`,
|
||||
}],
|
||||
isError: true,
|
||||
}
|
||||
}
|
||||
|
||||
const profile = await this.twitterService.getUserProfile(profileUsername)
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: `Profile Information:\n`
|
||||
+ `Username: @${profile.username}\n`
|
||||
+ `Display Name: ${profile.displayName}\n`
|
||||
+ `Bio: ${profile.bio || 'Not set'}\n`
|
||||
+ `Followers: ${profile.followersCount || 'N/A'}\n`
|
||||
+ `Following: ${profile.followingCount || 'N/A'}\n`
|
||||
+ `Tweets: ${profile.tweetCount || 'N/A'}\n`
|
||||
+ `Joined: ${profile.joinDate || 'N/A'}`,
|
||||
}],
|
||||
resources: [`twitter://user/${profile.username}`],
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Failed to get profile: ${errorToMessage(error)}` }],
|
||||
isError: true,
|
||||
}
|
||||
}
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract username from Twitter URL
|
||||
* @param url Twitter URL
|
||||
* @returns Username or undefined if not a profile URL
|
||||
*/
|
||||
private extractUsernameFromUrl(url: string): string | undefined {
|
||||
try {
|
||||
const parsedUrl = new URL(url)
|
||||
if (parsedUrl.hostname === 'x.com') {
|
||||
const pathParts = parsedUrl.pathname.split('/').filter(Boolean)
|
||||
if (pathParts.length > 0 && !['search', 'explore', 'home', 'notifications', 'messages'].includes(pathParts[0])) {
|
||||
return pathParts[0]
|
||||
}
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
catch (e) {
|
||||
logger.mcp.errorWithError('Error extracting username from URL:', e)
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Set up H3 routes
|
||||
*/
|
||||
private setupRoutes(): void {
|
||||
const router = createRouter()
|
||||
|
||||
// Set up CORS
|
||||
router.use('*', defineEventHandler((event) => {
|
||||
event.node.res.setHeader('Access-Control-Allow-Origin', '*')
|
||||
event.node.res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS')
|
||||
event.node.res.setHeader('Access-Control-Allow-Headers', 'Content-Type')
|
||||
|
||||
if (event.node.req.method === 'OPTIONS') {
|
||||
event.node.res.statusCode = 204
|
||||
event.node.res.end()
|
||||
}
|
||||
}))
|
||||
|
||||
// SSE endpoint
|
||||
router.get('/sse', defineEventHandler(async (event) => {
|
||||
const { req, res } = event.node
|
||||
|
||||
res.setHeader('Content-Type', 'text/event-stream')
|
||||
res.setHeader('Cache-Control', 'no-cache')
|
||||
res.setHeader('Connection', 'keep-alive')
|
||||
|
||||
// Create SSE transport
|
||||
const transport = new SSEServerTransport('/messages', res)
|
||||
this.activeTransports.push(transport)
|
||||
|
||||
// Clean up when client disconnects
|
||||
req.on('close', () => {
|
||||
const index = this.activeTransports.indexOf(transport)
|
||||
if (index !== -1) {
|
||||
this.activeTransports.splice(index, 1)
|
||||
}
|
||||
})
|
||||
|
||||
// Connect to MCP server
|
||||
await this.mcpServer.connect(transport)
|
||||
}))
|
||||
|
||||
// Messages endpoint - receive client requests
|
||||
router.post('/messages', defineEventHandler(async (event) => {
|
||||
if (this.activeTransports.length === 0) {
|
||||
logger.mcp.warn('Received message request but no active SSE connections')
|
||||
event.node.res.statusCode = 503
|
||||
return { error: 'No active SSE connections' }
|
||||
}
|
||||
|
||||
try {
|
||||
// Parse request body
|
||||
const body = await readBody(event)
|
||||
logger.mcp.debug(`Received MCP request: ${JSON.stringify(body)}`)
|
||||
|
||||
// Simple handling - send to most recent transport
|
||||
// Note: In production, should use session ID to route to correct transport
|
||||
const transport = this.activeTransports[this.activeTransports.length - 1]
|
||||
|
||||
// Manually handle POST message, as H3 is not Express-compatible
|
||||
const response = await transport.handleMessage(body)
|
||||
|
||||
// Log response for debugging
|
||||
logger.mcp.debug(`MCP response: ${JSON.stringify(response)}`)
|
||||
|
||||
return response
|
||||
}
|
||||
catch (error) {
|
||||
logger.mcp.errorWithError('Error handling MCP message:', error)
|
||||
event.node.res.statusCode = 500
|
||||
return { error: errorToMessage(error) }
|
||||
}
|
||||
}))
|
||||
|
||||
// Root path - provide service info
|
||||
router.get('/', defineEventHandler(() => {
|
||||
return {
|
||||
name: 'Twitter MCP Service',
|
||||
version: '1.0.0',
|
||||
endpoints: {
|
||||
sse: '/sse',
|
||||
messages: '/messages',
|
||||
},
|
||||
}
|
||||
}))
|
||||
|
||||
// Use router
|
||||
this.app.use(router)
|
||||
}
|
||||
|
||||
/**
|
||||
* Start MCP server
|
||||
*/
|
||||
start(): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
if (this.server !== null) {
|
||||
logger.mcp.warn('MCP server is already running')
|
||||
resolve()
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
// Create Node.js HTTP server
|
||||
this.server = createServer(toNodeListener(this.app))
|
||||
|
||||
// Add error event handlers
|
||||
this.server.on('error', (error) => {
|
||||
logger.mcp.errorWithError('MCP server error:', error)
|
||||
reject(error)
|
||||
})
|
||||
|
||||
// Log available resources for debugging
|
||||
logger.mcp.debug('Registered MCP resources:')
|
||||
logger.mcp.debug('- twitter://timeline/{count}: Get tweet timeline')
|
||||
logger.mcp.debug('- twitter://tweet/{id}: Get single tweet details')
|
||||
logger.mcp.debug('- twitter://user/{username}: Get user profile information')
|
||||
if (this.extraResourceInfo?.length) {
|
||||
this.extraResourceInfo.forEach((info) => {
|
||||
logger.mcp.debug(`- ${info}`)
|
||||
})
|
||||
}
|
||||
|
||||
this.server.listen(this.port, () => {
|
||||
const serverAddress = `http://localhost:${this.port}`
|
||||
logger.mcp.log(`MCP server started at: ${serverAddress}`)
|
||||
logger.mcp.log(`SSE endpoint: ${serverAddress}/sse`)
|
||||
logger.mcp.log(`Messages endpoint: ${serverAddress}/messages`)
|
||||
resolve()
|
||||
})
|
||||
}
|
||||
catch (error) {
|
||||
logger.mcp.errorWithError('Error starting MCP server:', error)
|
||||
reject(error)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop MCP server
|
||||
*/
|
||||
stop(): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
if (this.server === null) {
|
||||
logger.mcp.warn('MCP server is not running')
|
||||
resolve()
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
this.server.close(() => {
|
||||
this.server = null
|
||||
logger.mcp.log('MCP server stopped')
|
||||
resolve()
|
||||
})
|
||||
}
|
||||
catch (error) {
|
||||
logger.mcp.errorWithError('Error stopping MCP server:', error)
|
||||
this.server = null
|
||||
resolve()
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// h3 utility function: read body from event
|
||||
async function readBody(event: any): Promise<any> {
|
||||
const buffers = []
|
||||
for await (const chunk of event.node.req) {
|
||||
buffers.push(chunk)
|
||||
}
|
||||
const data = Buffer.concat(buffers).toString()
|
||||
return JSON.parse(data)
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
import type { Config } from './types'
|
||||
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import process from 'node:process'
|
||||
import { defu } from 'defu'
|
||||
import { config as configDotenv } from 'dotenv'
|
||||
|
||||
import { logger } from '../utils/logger'
|
||||
import { getDefaultConfig } from './types'
|
||||
|
||||
/**
|
||||
* Load environment variable files
|
||||
* Load in order of priority
|
||||
*/
|
||||
function loadEnvFiles(): void {
|
||||
// Load environment variable files
|
||||
const envFiles = [
|
||||
'.env.local',
|
||||
]
|
||||
|
||||
// Look for .env files from current directory upward
|
||||
for (const file of envFiles) {
|
||||
const filePath = path.resolve(process.cwd(), file)
|
||||
if (fs.existsSync(filePath)) {
|
||||
const result = configDotenv({
|
||||
path: filePath,
|
||||
override: true, // Allow overriding existing environment variables
|
||||
})
|
||||
|
||||
if (result.parsed) {
|
||||
logger.config.withFields({
|
||||
config: result.parsed,
|
||||
}).log(`Loaded environment variables from ${file}`)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Configuration manager
|
||||
* Responsible for loading, validating and providing configuration
|
||||
*/
|
||||
export class ConfigManager {
|
||||
private config: Config
|
||||
|
||||
/**
|
||||
* Create configuration manager
|
||||
* @param configPath Path to configuration file
|
||||
*/
|
||||
constructor(configPath?: string) {
|
||||
// First load environment variables
|
||||
loadEnvFiles()
|
||||
|
||||
// Set default configuration
|
||||
this.config = getDefaultConfig()
|
||||
|
||||
// Then load from configuration file (if specified)
|
||||
if (configPath) {
|
||||
this.loadFromFile(configPath)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Load configuration from file
|
||||
*/
|
||||
private loadFromFile(filePath: string): void {
|
||||
try {
|
||||
const configFile = fs.readFileSync(filePath, 'utf8')
|
||||
const fileConfig = JSON.parse(configFile)
|
||||
|
||||
// Use defu to deeply merge configurations
|
||||
// Values in fileConfig take precedence over this.config
|
||||
this.config = defu(fileConfig, this.config)
|
||||
|
||||
logger.config.log(`Configuration loaded from ${filePath}`)
|
||||
}
|
||||
catch (error) {
|
||||
logger.config.errorWithError(`Failed to load configuration file: ${(error as Error).message}`, error)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get complete configuration
|
||||
*/
|
||||
getConfig(): Config {
|
||||
return this.config
|
||||
}
|
||||
|
||||
/**
|
||||
* Update configuration
|
||||
*/
|
||||
updateConfig(newConfig: Partial<Config>): void {
|
||||
// Use defu to merge new configuration
|
||||
this.config = defu(newConfig, this.config)
|
||||
}
|
||||
}
|
||||
|
||||
// Singleton instance
|
||||
let configInstance: ConfigManager | null = null
|
||||
|
||||
/**
|
||||
* Create default configuration manager (singleton)
|
||||
*/
|
||||
export function useConfigManager(): ConfigManager {
|
||||
if (configInstance) {
|
||||
return configInstance
|
||||
}
|
||||
|
||||
const configPath = process.env.CONFIG_PATH || path.join(process.cwd(), 'twitter-config.json')
|
||||
configInstance = new ConfigManager(fs.existsSync(configPath) ? configPath : undefined)
|
||||
return configInstance
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
import type { BrowserConfig } from '../types/browser'
|
||||
import type { SearchOptions, TimelineOptions } from '../types/twitter'
|
||||
|
||||
import process from 'node:process'
|
||||
|
||||
/**
|
||||
* Complete configuration interface
|
||||
*/
|
||||
export interface Config {
|
||||
// Browser configuration
|
||||
browser: BrowserConfig & {
|
||||
apiKey: string // API Key for Stagehand
|
||||
endpoint?: string // Optional Stagehand service endpoint
|
||||
}
|
||||
|
||||
// Twitter configuration
|
||||
twitter: {
|
||||
defaultOptions?: {
|
||||
timeline?: TimelineOptions
|
||||
search?: SearchOptions
|
||||
}
|
||||
}
|
||||
|
||||
// Adapter configuration
|
||||
adapters: {
|
||||
airi?: {
|
||||
url?: string
|
||||
token?: string
|
||||
enabled: boolean
|
||||
}
|
||||
mcp?: {
|
||||
port?: number
|
||||
enabled: boolean
|
||||
}
|
||||
}
|
||||
|
||||
// System configuration
|
||||
system: {
|
||||
logLevel: 'error' | 'warn' | 'info' | 'verbose' | 'debug'
|
||||
logFormat?: 'json' | 'pretty'
|
||||
concurrency: number
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Default configuration
|
||||
*/
|
||||
export function getDefaultConfig(): Config {
|
||||
// No longer parse cookies from environment variable
|
||||
// The auth service will load cookies from session file instead
|
||||
|
||||
return {
|
||||
browser: {
|
||||
apiKey: process.env.BROWSERBASE_API_KEY || '', // Move apiKey to browser config
|
||||
headless: process.env.BROWSER_HEADLESS === 'true',
|
||||
userAgent: process.env.BROWSER_USER_AGENT || 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
|
||||
viewport: {
|
||||
width: Number.parseInt(process.env.BROWSER_VIEWPORT_WIDTH || '1280'),
|
||||
height: Number.parseInt(process.env.BROWSER_VIEWPORT_HEIGHT || '800'),
|
||||
},
|
||||
timeout: Number.parseInt(process.env.BROWSER_TIMEOUT || '30000'),
|
||||
requestTimeout: Number.parseInt(process.env.BROWSER_REQUEST_TIMEOUT || '20000'),
|
||||
requestRetries: Number.parseInt(process.env.BROWSER_REQUEST_RETRIES || '2'),
|
||||
},
|
||||
twitter: {
|
||||
defaultOptions: {
|
||||
timeline: {
|
||||
count: 20,
|
||||
includeReplies: true,
|
||||
includeRetweets: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
adapters: {
|
||||
airi: {
|
||||
url: process.env.AIRI_URL || 'http://localhost:3000',
|
||||
token: process.env.AIRI_TOKEN || '',
|
||||
enabled: process.env.ENABLE_AIRI === 'true',
|
||||
},
|
||||
mcp: {
|
||||
port: Number(process.env.MCP_PORT || 8080),
|
||||
enabled: process.env.ENABLE_MCP === 'true' || true,
|
||||
},
|
||||
},
|
||||
system: {
|
||||
logLevel: 'debug',
|
||||
logFormat: 'pretty',
|
||||
concurrency: Number(process.env.CONCURRENCY || 1),
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,614 @@
|
||||
import type { BrowserContext, Cookie, Page } from 'playwright'
|
||||
|
||||
import fs from 'node:fs/promises'
|
||||
import path from 'node:path'
|
||||
import process from 'node:process'
|
||||
|
||||
import { logger } from '../utils/logger'
|
||||
import { SELECTORS } from '../utils/selectors'
|
||||
|
||||
/**
|
||||
* Playwright storage state type definition
|
||||
*/
|
||||
interface StorageState {
|
||||
cookies: Cookie[]
|
||||
origins: {
|
||||
origin: string
|
||||
localStorage: {
|
||||
name: string
|
||||
value: string
|
||||
}[]
|
||||
}[]
|
||||
path?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Simple session manager for storing and retrieving browser session data
|
||||
*/
|
||||
class SessionManager {
|
||||
private sessionPath: string
|
||||
|
||||
constructor() {
|
||||
this.sessionPath = path.join(process.cwd(), 'data', 'twitter-session.json')
|
||||
}
|
||||
|
||||
/**
|
||||
* Load storage state from disk
|
||||
*/
|
||||
async loadStorageState(): Promise<StorageState | null> {
|
||||
try {
|
||||
// Ensure directory exists
|
||||
const dir = path.dirname(this.sessionPath)
|
||||
await fs.mkdir(dir, { recursive: true })
|
||||
|
||||
// Check if file exists
|
||||
try {
|
||||
await fs.access(this.sessionPath)
|
||||
}
|
||||
catch {
|
||||
// File doesn't exist
|
||||
return null
|
||||
}
|
||||
|
||||
// Read file
|
||||
const data = await fs.readFile(this.sessionPath, 'utf-8')
|
||||
return JSON.parse(data) as StorageState
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).warn('Failed to load session data')
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Save storage state to disk
|
||||
*/
|
||||
async saveStorageState(state: StorageState): Promise<void> {
|
||||
try {
|
||||
// Ensure directory exists
|
||||
const dir = path.dirname(this.sessionPath)
|
||||
await fs.mkdir(dir, { recursive: true })
|
||||
|
||||
// Write to file
|
||||
await fs.writeFile(this.sessionPath, JSON.stringify(state, null, 2))
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).warn('Failed to save session data')
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Singleton instance
|
||||
const sessionManager = new SessionManager()
|
||||
|
||||
/**
|
||||
* Twitter Authentication Service
|
||||
* Handles login and session management
|
||||
*/
|
||||
export class TwitterAuthService {
|
||||
private page: Page
|
||||
private context: BrowserContext
|
||||
private isLoggedIn: boolean = false
|
||||
|
||||
constructor(page: Page, context: BrowserContext) {
|
||||
this.page = page
|
||||
this.context = context
|
||||
}
|
||||
|
||||
/**
|
||||
* Login to Twitter - simplified method that only tries to use session file
|
||||
* Users are expected to manually login and save the session
|
||||
*/
|
||||
async login(): Promise<boolean> {
|
||||
logger.auth.log('Starting Twitter login process')
|
||||
|
||||
try {
|
||||
// Try to login with existing session first
|
||||
logger.auth.log('Attempting to load session from file')
|
||||
const sessionSuccess = await this.checkExistingSession()
|
||||
|
||||
if (sessionSuccess) {
|
||||
logger.auth.log('Successfully logged in with session file')
|
||||
return true
|
||||
}
|
||||
|
||||
// Log session failure but don't attempt automatic login
|
||||
logger.auth.log('No valid session found, manual login is required')
|
||||
return false
|
||||
}
|
||||
catch (error: unknown) {
|
||||
logger.auth.withError(error as Error).error('Login process failed')
|
||||
this.isLoggedIn = false
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify if login was successful
|
||||
*/
|
||||
private async verifyLogin(): Promise<boolean> {
|
||||
try {
|
||||
// Try multiple selectors to determine login status
|
||||
// First check for timeline which is definitive proof of being logged in
|
||||
try {
|
||||
await this.page.waitForSelector(SELECTORS.HOME.TIMELINE, { timeout: 15000 })
|
||||
|
||||
// Login verification successful - automatically save session
|
||||
this.isLoggedIn = true
|
||||
try {
|
||||
await this.saveCurrentSession()
|
||||
logger.auth.log('✅ Auto-saved session after successful login verification')
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).warn('Failed to auto-save session')
|
||||
}
|
||||
|
||||
return true
|
||||
}
|
||||
catch {
|
||||
// If timeline selector fails, check for other indicators
|
||||
}
|
||||
|
||||
// Check for profile button which appears when logged in
|
||||
try {
|
||||
const profileSelector = '[data-testid="AppTabBar_Profile_Link"]'
|
||||
await this.page.waitForSelector(profileSelector, { timeout: 5000 })
|
||||
|
||||
// Profile link found - automatically save session
|
||||
this.isLoggedIn = true
|
||||
try {
|
||||
await this.saveCurrentSession()
|
||||
logger.auth.log('✅ Auto-saved session after finding profile link')
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).warn('Failed to auto-save session')
|
||||
}
|
||||
|
||||
return true
|
||||
}
|
||||
catch {
|
||||
// Continue to other checks
|
||||
}
|
||||
|
||||
// Check for login form to confirm NOT logged in
|
||||
try {
|
||||
const loginFormSelector = '[data-testid="loginForm"]'
|
||||
await this.page.waitForSelector(loginFormSelector, { timeout: 3000 })
|
||||
// If login form is visible, we're definitely not logged in
|
||||
return false
|
||||
}
|
||||
catch {
|
||||
// Login form not found, could still be logged in or on another page
|
||||
}
|
||||
|
||||
// If we got here, we couldn't definitively confirm login status
|
||||
// Check current URL for additional clues
|
||||
const currentUrl = await this.page.evaluate<string>(`
|
||||
(() => {
|
||||
return window.location.href;
|
||||
})()
|
||||
`)
|
||||
if (currentUrl.includes('/home')) {
|
||||
// On home page but couldn't find timeline - might still be loading
|
||||
return true
|
||||
}
|
||||
|
||||
// Default to not logged in if we can't confirm
|
||||
return false
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).error('Error during login verification')
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check current login status
|
||||
*/
|
||||
async checkLoginStatus(): Promise<boolean> {
|
||||
try {
|
||||
await this.page.goto('https://x.com/home')
|
||||
const isLoggedIn = await this.verifyLogin()
|
||||
|
||||
// If already logged in, update state and automatically save session
|
||||
if (isLoggedIn && !this.isLoggedIn) {
|
||||
this.isLoggedIn = true
|
||||
try {
|
||||
await this.saveCurrentSession()
|
||||
logger.auth.log('✅ Auto-saved session during status check')
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).warn('Failed to auto-save session during status check')
|
||||
}
|
||||
}
|
||||
|
||||
return isLoggedIn
|
||||
}
|
||||
catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get login status
|
||||
*/
|
||||
isAuthenticated(): boolean {
|
||||
return this.isLoggedIn
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current page URL
|
||||
* @returns Current URL of the Twitter page
|
||||
*/
|
||||
async getCurrentUrl(): Promise<string> {
|
||||
try {
|
||||
const currentUrl = await this.page.evaluate<string>(`
|
||||
(() => {
|
||||
return window.location.href;
|
||||
})()
|
||||
`)
|
||||
return currentUrl
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).error('Error getting current URL')
|
||||
throw new Error('Failed to get current URL')
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Export cookies from the browser context
|
||||
* @param format - The format of the returned cookies ('object' or 'string')
|
||||
*/
|
||||
async exportCookies(format: 'object' | 'string' = 'object'): Promise<Record<string, string> | string> {
|
||||
try {
|
||||
// Get all cookies from browser
|
||||
const allCookies = await this.context.cookies()
|
||||
|
||||
if (format === 'string') {
|
||||
// Convert cookie objects to string format
|
||||
const cookieString = allCookies
|
||||
.map(cookie => `${cookie.name}=${cookie.value}`)
|
||||
.join('; ')
|
||||
|
||||
logger.auth.log(`Exported ${allCookies.length} cookies as string`)
|
||||
return cookieString
|
||||
}
|
||||
else {
|
||||
// Convert to object format
|
||||
const cookiesObj = allCookies.reduce<Record<string, string>>((acc, cookie) => {
|
||||
acc[cookie.name] = cookie.value
|
||||
return acc
|
||||
}, {})
|
||||
|
||||
logger.auth.log(`Exported ${Object.keys(cookiesObj).length} cookies as object`)
|
||||
return cookiesObj
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).error('Failed to export cookies')
|
||||
if (format === 'string') {
|
||||
return ''
|
||||
}
|
||||
return {}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Login to Twitter using cookies
|
||||
*/
|
||||
async loginWithCookies(cookies: Record<string, string>): Promise<boolean> {
|
||||
logger.auth.log(`Attempting to login to Twitter using ${Object.keys(cookies).length} cookies`)
|
||||
|
||||
try {
|
||||
// Navigate to a Twitter page
|
||||
await this.page.goto('https://x.com')
|
||||
|
||||
// Convert cookies object to array format required by setCookies
|
||||
const cookieArray = Object.entries(cookies).map(([name, value]) => ({
|
||||
name,
|
||||
value,
|
||||
domain: '.x.com',
|
||||
path: '/',
|
||||
}))
|
||||
|
||||
// Set cookies using the browser adapter's API that can set HTTP_ONLY cookies
|
||||
await this.context.addCookies(cookieArray)
|
||||
|
||||
logger.auth.log(`Set ${cookieArray.length} cookies via browser API`)
|
||||
|
||||
// Refresh page to apply cookies
|
||||
await this.page.goto('https://x.com/home')
|
||||
|
||||
// Verify if login was successful - try multiple times with longer timeout
|
||||
logger.auth.log('Cookies set, verifying login status...')
|
||||
|
||||
// Try multiple times with increasing timeouts for verification
|
||||
// Twitter might be slow to respond or need multiple page refreshes
|
||||
let loginSuccess = false
|
||||
const verificationAttempts = 3
|
||||
|
||||
for (let attempt = 1; attempt <= verificationAttempts; attempt++) {
|
||||
try {
|
||||
logger.auth.log(`Verification attempt ${attempt}/${verificationAttempts}`)
|
||||
loginSuccess = await this.verifyLogin()
|
||||
|
||||
if (loginSuccess) {
|
||||
break
|
||||
}
|
||||
else if (attempt < verificationAttempts) {
|
||||
// If not successful but not last attempt, refresh page and wait
|
||||
logger.auth.log('Refreshing page and trying again...')
|
||||
await this.page.goto('https://x.com/home')
|
||||
await new Promise(resolve => setTimeout(resolve, 3000))
|
||||
}
|
||||
}
|
||||
catch (error: unknown) {
|
||||
logger.auth.withError(error as Error).debug(`Verification attempt ${attempt} failed`)
|
||||
}
|
||||
}
|
||||
|
||||
if (loginSuccess) {
|
||||
logger.auth.log('Login with cookies successful')
|
||||
this.isLoggedIn = true
|
||||
|
||||
// Try to refresh cookies to ensure they're up to date
|
||||
try {
|
||||
await this.saveCurrentSession()
|
||||
logger.auth.log('✅ Session saved to file')
|
||||
}
|
||||
catch (error: unknown) {
|
||||
logger.auth.withError(error as Error).debug('Failed to save session, but login was successful')
|
||||
}
|
||||
}
|
||||
else {
|
||||
logger.auth.warn('Login with cookies verification failed, cookies may be expired')
|
||||
}
|
||||
|
||||
return loginSuccess
|
||||
}
|
||||
catch (error: unknown) {
|
||||
logger.auth.withError(error as Error).error('Error during cookie login process')
|
||||
this.isLoggedIn = false
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Attempt to login with an existing session if available
|
||||
*/
|
||||
async checkExistingSession(): Promise<boolean> {
|
||||
try {
|
||||
// Get the session data
|
||||
const sessionData = await sessionManager.loadStorageState()
|
||||
|
||||
if (!sessionData || !sessionData.cookies || sessionData.cookies.length === 0) {
|
||||
logger.auth.log('No valid session data found')
|
||||
return false
|
||||
}
|
||||
|
||||
logger.auth.log(`Found session file with ${sessionData.cookies.length} cookies, attempting login`)
|
||||
|
||||
// Login with the session data
|
||||
return await this.loginWithSessionData(sessionData)
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).warn('Error checking existing session')
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Initiate manual login process with username and password
|
||||
* @param username Twitter username or email
|
||||
* @param password Twitter password
|
||||
*/
|
||||
async initiateManualLogin(username?: string, password?: string): Promise<boolean> {
|
||||
logger.auth.log('Initiating manual login process')
|
||||
|
||||
try {
|
||||
// Navigate to login page
|
||||
await this.page.goto('https://x.com/login')
|
||||
|
||||
// Wait for login form to appear and enter credentials
|
||||
try {
|
||||
// Wait for username input
|
||||
await this.page.waitForSelector(SELECTORS.LOGIN.USERNAME_INPUT, { timeout: 10000 })
|
||||
|
||||
// Use provided credentials if available, otherwise fall back to env vars
|
||||
const loginUsername = username || process.env.TWITTER_USERNAME
|
||||
const loginPassword = password || process.env.TWITTER_PASSWORD
|
||||
|
||||
if (!loginUsername || !loginPassword) {
|
||||
logger.auth.warn('Missing Twitter credentials, manual login cannot proceed')
|
||||
return false
|
||||
}
|
||||
|
||||
// Enter username
|
||||
await this.page.fill(SELECTORS.LOGIN.USERNAME_INPUT, loginUsername)
|
||||
logger.auth.debug('Username entered')
|
||||
|
||||
// Click next button
|
||||
await this.page.click(SELECTORS.LOGIN.NEXT_BUTTON)
|
||||
logger.auth.debug('Next button clicked')
|
||||
|
||||
// Wait for password input
|
||||
await this.page.waitForSelector(SELECTORS.LOGIN.PASSWORD_INPUT, { timeout: 10000 })
|
||||
|
||||
// Enter password
|
||||
await this.page.fill(SELECTORS.LOGIN.PASSWORD_INPUT, loginPassword)
|
||||
logger.auth.debug('Password entered')
|
||||
|
||||
// Click login button
|
||||
await this.page.click(SELECTORS.LOGIN.LOGIN_BUTTON)
|
||||
logger.auth.debug('Login button clicked')
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).error('Error during manual login process')
|
||||
return false
|
||||
}
|
||||
|
||||
// Wait for login success at intervals
|
||||
let attempts = 0
|
||||
const maxAttempts = 60 // 10 minutes (10 seconds * 60)
|
||||
let lastUrl = await this.page.evaluate<string>(`
|
||||
(() => {
|
||||
return window.location.href;
|
||||
})()
|
||||
`)
|
||||
|
||||
while (attempts < maxAttempts) {
|
||||
attempts++
|
||||
|
||||
try {
|
||||
// Get current URL to detect page changes
|
||||
const currentUrl = await this.page.evaluate<string>(`
|
||||
(() => {
|
||||
return window.location.href;
|
||||
})()
|
||||
`)
|
||||
|
||||
// Check if URL has changed significantly - may indicate user interaction
|
||||
if (currentUrl !== lastUrl && !currentUrl.includes('/flow/login')) {
|
||||
logger.auth.log(`Detected page change: ${lastUrl} -> ${currentUrl}`)
|
||||
logger.auth.log('Attempting to navigate to home page and verify login status')
|
||||
|
||||
// URL changed - try navigating to home to verify
|
||||
await this.page.goto('https://x.com/home')
|
||||
|
||||
// Check if login was successful
|
||||
const isLoggedIn = await this.verifyLogin()
|
||||
if (isLoggedIn) {
|
||||
logger.auth.log('✅ Login successful! Exporting cookies...')
|
||||
|
||||
// Export cookies for future use
|
||||
try {
|
||||
const cookies = await this.exportCookies('object')
|
||||
logger.auth.log(`✅ Successfully exported ${typeof cookies === 'string' ? cookies.length : Object.keys(cookies).length} cookies`)
|
||||
|
||||
// Save the current session to file
|
||||
await this.saveCurrentSession()
|
||||
logger.auth.log('✅ Session saved to file')
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).error('Error exporting cookies')
|
||||
}
|
||||
|
||||
this.isLoggedIn = true
|
||||
return true
|
||||
}
|
||||
|
||||
// Update last URL
|
||||
lastUrl = currentUrl
|
||||
}
|
||||
|
||||
// Also try direct login verification
|
||||
const isLoggedIn = await this.verifyLogin()
|
||||
if (isLoggedIn) {
|
||||
logger.auth.log('✅ Login successful! Exporting cookies...')
|
||||
|
||||
// Export cookies for future use
|
||||
try {
|
||||
const cookies = await this.exportCookies('object')
|
||||
logger.auth.log(`✅ Successfully exported ${typeof cookies === 'string' ? cookies.length : Object.keys(cookies).length} cookies`)
|
||||
|
||||
// Save the current session to file
|
||||
await this.saveCurrentSession()
|
||||
logger.auth.log('✅ Session saved to file')
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).error('Error exporting cookies')
|
||||
}
|
||||
|
||||
this.isLoggedIn = true
|
||||
return true
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
// Ignore errors during verification, continue polling
|
||||
logger.auth.debug(`Error during verification: ${(error as Error).message}`)
|
||||
}
|
||||
|
||||
// Wait 10 seconds before checking again
|
||||
await new Promise(resolve => setTimeout(resolve, 10000))
|
||||
|
||||
// Only log every 6 attempts (1 minute) to reduce noise
|
||||
if (attempts % 6 === 0) {
|
||||
logger.auth.log(`Still waiting for login... (${Math.floor(attempts / 6)} minutes elapsed)`)
|
||||
}
|
||||
}
|
||||
|
||||
logger.auth.warn('⚠️ Manual login timeout exceeded')
|
||||
return false
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).error('Error during manual login process')
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Save the current session to a file
|
||||
*/
|
||||
async saveCurrentSession(): Promise<void> {
|
||||
try {
|
||||
// Get the storage state directly from context
|
||||
const storageState = await this.context.storageState()
|
||||
|
||||
// Save the session using the session manager
|
||||
await sessionManager.saveStorageState(storageState)
|
||||
|
||||
logger.auth.log('✅ Session saved to file using browserContext.storageState()')
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).warn('Failed to save session')
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Login with stored session data
|
||||
*/
|
||||
private async loginWithSessionData(sessionData: StorageState): Promise<boolean> {
|
||||
try {
|
||||
// Extract cookies from session data
|
||||
const { cookies } = sessionData
|
||||
|
||||
// Create array of cookie objects for the browser
|
||||
const cookieArray = cookies.map(cookie => ({
|
||||
name: cookie.name,
|
||||
value: cookie.value,
|
||||
domain: cookie.domain,
|
||||
path: cookie.path,
|
||||
}))
|
||||
|
||||
// Set cookies using the browser adapter's API
|
||||
await this.context.addCookies(cookieArray)
|
||||
logger.auth.log(`Set ${cookieArray.length} cookies from session file`)
|
||||
|
||||
// Set localStorage if available
|
||||
if (sessionData.origins && sessionData.origins.length > 0) {
|
||||
await this.context.storageState(sessionData)
|
||||
logger.auth.log(`Set localStorage for ${sessionData.origins.length} origins`)
|
||||
}
|
||||
|
||||
// Navigate to home to verify login
|
||||
await this.page.goto('https://x.com/home')
|
||||
|
||||
// Verify if login was successful
|
||||
const loginSuccess = await this.verifyLogin()
|
||||
|
||||
if (loginSuccess) {
|
||||
this.isLoggedIn = true
|
||||
logger.auth.log('✅ Successfully logged in with session data')
|
||||
}
|
||||
else {
|
||||
logger.auth.warn('⚠️ Session data login failed verification')
|
||||
}
|
||||
|
||||
return loginSuccess
|
||||
}
|
||||
catch (error) {
|
||||
logger.auth.withError(error as Error).error('Failed to login with session data')
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
import type { Page } from 'playwright'
|
||||
import type { TimelineOptions, Tweet } from '../types/twitter'
|
||||
|
||||
import { TweetParser } from '../parsers/tweet-parser'
|
||||
import { logger } from '../utils/logger'
|
||||
import { SELECTORS } from '../utils/selectors'
|
||||
|
||||
/**
|
||||
* Twitter Timeline Service
|
||||
* Handles fetching and parsing timeline content
|
||||
*/
|
||||
export class TwitterTimelineService {
|
||||
private page: Page
|
||||
|
||||
constructor(page: Page) {
|
||||
this.page = page
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetches the Twitter timeline
|
||||
* @param options Configuration options for timeline fetching
|
||||
* @returns Promise resolving to an array of tweets
|
||||
*/
|
||||
async getTimeline(options: TimelineOptions = {}): Promise<Tweet[]> {
|
||||
try {
|
||||
logger.timeline.withFields({ options }).log('Fetching timeline')
|
||||
|
||||
// Navigate to home page
|
||||
await this.page.goto('https://x.com/home')
|
||||
|
||||
// Wait for timeline to load
|
||||
await this.page.waitForSelector(SELECTORS.TIMELINE.TWEET, { timeout: 10000 })
|
||||
|
||||
// Optional: scroll to load more tweets if needed
|
||||
if (options.count && options.count > 5) {
|
||||
await this.scrollToLoadMoreTweets(Math.min(options.count, 20))
|
||||
}
|
||||
|
||||
// Parse all tweets directly from the DOM using Playwright
|
||||
const tweets = await TweetParser.parseTimelineTweets(this.page)
|
||||
|
||||
logger.timeline.log(`Found ${tweets.length} tweets in timeline`)
|
||||
|
||||
// Apply filters
|
||||
let filteredTweets = tweets
|
||||
|
||||
if (options.includeReplies === false) {
|
||||
filteredTweets = filteredTweets.filter(tweet => !tweet.text.startsWith('@'))
|
||||
}
|
||||
|
||||
if (options.includeRetweets === false) {
|
||||
filteredTweets = filteredTweets.filter(tweet => !tweet.text.startsWith('RT @'))
|
||||
}
|
||||
|
||||
// Apply count limit if specified
|
||||
if (options.count) {
|
||||
filteredTweets = filteredTweets.slice(0, options.count)
|
||||
}
|
||||
|
||||
return filteredTweets
|
||||
}
|
||||
catch (error) {
|
||||
logger.timeline.error('Failed to get timeline:', (error as Error).message)
|
||||
return []
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Scrolls down the timeline to load more tweets
|
||||
* @param targetCount Approximate number of tweets to load
|
||||
*/
|
||||
private async scrollToLoadMoreTweets(targetCount: number): Promise<void> {
|
||||
try {
|
||||
// Initial tweet count
|
||||
let previousTweetCount = 0
|
||||
let currentTweetCount = await this.countVisibleTweets()
|
||||
let scrollAttempts = 0
|
||||
const maxScrollAttempts = 10
|
||||
|
||||
logger.timeline.log(`Initial tweet count: ${currentTweetCount}, target: ${targetCount}`)
|
||||
|
||||
// Scroll until we have enough tweets or reach maximum scroll attempts
|
||||
while (currentTweetCount < targetCount && scrollAttempts < maxScrollAttempts) {
|
||||
// Scroll down using Playwright's mouse wheel simulation
|
||||
await this.page.mouse.wheel(0, 800)
|
||||
|
||||
// Wait for new content to load
|
||||
await this.page.waitForTimeout(1000)
|
||||
|
||||
// Check if we have new tweets
|
||||
previousTweetCount = currentTweetCount
|
||||
currentTweetCount = await this.countVisibleTweets()
|
||||
|
||||
// If no new tweets were loaded, we might have reached the end
|
||||
if (currentTweetCount === previousTweetCount) {
|
||||
scrollAttempts++
|
||||
}
|
||||
else {
|
||||
scrollAttempts = 0 // Reset counter if we're still loading tweets
|
||||
}
|
||||
|
||||
logger.timeline.debug(`Scrolled for more tweets: ${currentTweetCount}/${targetCount}`)
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
logger.timeline.error('Error while scrolling for more tweets:', (error as Error).message)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Counts the number of visible tweets on the page
|
||||
* @returns Promise resolving to the count of visible tweets
|
||||
*/
|
||||
private async countVisibleTweets(): Promise<number> {
|
||||
const tweetElements = await this.page.$$(SELECTORS.TIMELINE.TWEET)
|
||||
return tweetElements.length
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,222 @@
|
||||
import type { PostOptions, SearchOptions, TimelineOptions, Tweet, TweetDetail, UserProfile } from '../types/twitter'
|
||||
import type { TwitterAuthService } from './auth-service'
|
||||
import type { TwitterTimelineService } from './timeline-service'
|
||||
|
||||
import { logger } from '../utils/logger'
|
||||
|
||||
export class TwitterService {
|
||||
private authService: TwitterAuthService
|
||||
private timelineService: TwitterTimelineService
|
||||
private sessionMonitorInterval: NodeJS.Timeout | null = null
|
||||
|
||||
constructor(authService: TwitterAuthService, timelineService: TwitterTimelineService) {
|
||||
this.authService = authService
|
||||
this.timelineService = timelineService
|
||||
}
|
||||
|
||||
/**
|
||||
* Login to Twitter
|
||||
* Attempts to restore session from saved cookies first
|
||||
* If that fails, will need manual login in the browser
|
||||
*/
|
||||
async login(): Promise<boolean> {
|
||||
try {
|
||||
// Try to restore session from cookies
|
||||
const success = await this.authService.login()
|
||||
|
||||
if (success) {
|
||||
logger.main.log('Successfully restored Twitter session from cookies')
|
||||
return true
|
||||
}
|
||||
|
||||
logger.main.log('No saved session found, waiting for manual login')
|
||||
|
||||
// Set up session monitoring to detect when user has manually logged in
|
||||
this.startSessionMonitor()
|
||||
|
||||
return false
|
||||
}
|
||||
catch (error) {
|
||||
logger.main.error('Error during login:', (error as Error).message)
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get timeline
|
||||
*/
|
||||
async getTimeline(options?: TimelineOptions): Promise<Tweet[]> {
|
||||
this.ensureAuthenticated()
|
||||
return this.timelineService.getTimeline(options)
|
||||
}
|
||||
|
||||
/**
|
||||
* Get tweet details
|
||||
*/
|
||||
async getTweetDetails(tweetId: string): Promise<TweetDetail> {
|
||||
this.ensureAuthenticated()
|
||||
// This is a stub implementation
|
||||
return {
|
||||
id: tweetId,
|
||||
text: 'Tweet details feature not yet implemented',
|
||||
author: {
|
||||
username: 'twitter',
|
||||
displayName: 'Twitter',
|
||||
},
|
||||
timestamp: new Date().toISOString(),
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Search tweets
|
||||
*/
|
||||
async searchTweets(_query: string, _options?: SearchOptions): Promise<Tweet[]> {
|
||||
throw new Error('Search feature not yet implemented')
|
||||
}
|
||||
|
||||
/**
|
||||
* Get user profile
|
||||
*/
|
||||
async getUserProfile(_username: string): Promise<UserProfile> {
|
||||
throw new Error('Get user profile feature not yet implemented')
|
||||
}
|
||||
|
||||
/**
|
||||
* Follow user (not implemented in MVP)
|
||||
*/
|
||||
async followUser(_username: string): Promise<boolean> {
|
||||
this.ensureAuthenticated()
|
||||
return false
|
||||
}
|
||||
|
||||
/**
|
||||
* Like tweet
|
||||
*/
|
||||
async likeTweet(_tweetId: string): Promise<boolean> {
|
||||
throw new Error('Like feature not yet implemented')
|
||||
}
|
||||
|
||||
/**
|
||||
* Retweet
|
||||
*/
|
||||
async retweet(_tweetId: string): Promise<boolean> {
|
||||
throw new Error('Retweet feature not yet implemented')
|
||||
}
|
||||
|
||||
/**
|
||||
* Post a tweet
|
||||
*/
|
||||
async postTweet(_content: string, _options?: PostOptions): Promise<string> {
|
||||
throw new Error('Post tweet feature not yet implemented')
|
||||
}
|
||||
|
||||
/**
|
||||
* Manually trigger a session save
|
||||
* Typically this is handled by the session monitor
|
||||
*/
|
||||
async saveSession(): Promise<boolean> {
|
||||
try {
|
||||
if (!this.authService.isAuthenticated()) {
|
||||
logger.main.warn('Cannot save session when not authenticated')
|
||||
return false
|
||||
}
|
||||
|
||||
await this.authService.saveCurrentSession()
|
||||
logger.main.log('Successfully saved Twitter session')
|
||||
return true
|
||||
}
|
||||
catch (error) {
|
||||
logger.main.error('Error saving session:', (error as Error).message)
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure the user is authenticated before performing operations
|
||||
* @private
|
||||
*/
|
||||
private ensureAuthenticated(): void {
|
||||
if (!this.authService.isAuthenticated()) {
|
||||
throw new Error('You must be logged in to perform this action. Please call login() first.')
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Export the current session cookies
|
||||
* @param format The format to export cookies in
|
||||
*/
|
||||
async exportCookies(format: 'object' | 'string' = 'object'): Promise<Record<string, string> | string> {
|
||||
this.ensureAuthenticated()
|
||||
return this.authService.exportCookies(format)
|
||||
}
|
||||
|
||||
/**
|
||||
* Start monitoring for session changes
|
||||
* This will periodically check if the user is logged in
|
||||
* and save the session if they are
|
||||
* @param interval Time in ms between checks
|
||||
*/
|
||||
startSessionMonitor(interval: number = 30000): void {
|
||||
// Clear any existing monitor
|
||||
if (this.sessionMonitorInterval) {
|
||||
clearInterval(this.sessionMonitorInterval)
|
||||
}
|
||||
|
||||
logger.main.log(`Starting Twitter session monitor with ${interval}ms interval`)
|
||||
|
||||
this.sessionMonitorInterval = setInterval(async () => {
|
||||
try {
|
||||
await this.checkAndSaveSession()
|
||||
}
|
||||
catch (error) {
|
||||
logger.main.error('Error in session monitor:', (error as Error).message)
|
||||
}
|
||||
}, interval)
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the current page URL
|
||||
* Useful for debugging and checking the current state
|
||||
*/
|
||||
async getCurrentUrl(): Promise<string> {
|
||||
try {
|
||||
return await this.authService.getCurrentUrl()
|
||||
}
|
||||
catch (error) {
|
||||
logger.main.error('Error getting current URL:', (error as Error).message)
|
||||
return 'unknown'
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop the session monitor
|
||||
*/
|
||||
stopSessionMonitor(): void {
|
||||
if (this.sessionMonitorInterval) {
|
||||
clearInterval(this.sessionMonitorInterval)
|
||||
this.sessionMonitorInterval = null
|
||||
logger.main.log('Stopped Twitter session monitor')
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check and save the session if logged in
|
||||
* @private
|
||||
*/
|
||||
private async checkAndSaveSession(): Promise<void> {
|
||||
try {
|
||||
const isLoggedIn = await this.authService.checkLoginStatus()
|
||||
|
||||
if (isLoggedIn && !this.authService.isAuthenticated()) {
|
||||
logger.main.log('User has logged in manually, saving session')
|
||||
await this.saveSession()
|
||||
}
|
||||
else if (!isLoggedIn && this.authService.isAuthenticated()) {
|
||||
logger.main.warn('User appears to be logged out, updating state')
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
logger.main.error('Error checking session status:', (error as Error).message)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,188 @@
|
||||
import type { Browser, BrowserContext, Page } from 'playwright'
|
||||
import type { AiriAdapter } from './adapters/airi-adapter'
|
||||
import type { MCPAdapter } from './adapters/mcp-adapter'
|
||||
import type { Config } from './config/types'
|
||||
|
||||
import process from 'node:process'
|
||||
import { chromium } from 'playwright'
|
||||
|
||||
import { useConfigManager } from './config'
|
||||
import { TwitterAuthService } from './core/auth-service'
|
||||
import { TwitterTimelineService } from './core/timeline-service'
|
||||
import { TwitterService } from './core/twitter-service'
|
||||
import { initLogger, logger } from './utils/logger'
|
||||
|
||||
/**
|
||||
* Initialize browser and create page
|
||||
*/
|
||||
async function initBrowser(config: Config): Promise<{ browser: Browser, context: BrowserContext, page: Page }> {
|
||||
const browser = await chromium.launch({
|
||||
headless: config.browser.headless,
|
||||
})
|
||||
|
||||
const context = await browser.newContext({
|
||||
userAgent: config.browser.userAgent,
|
||||
viewport: config.browser.viewport,
|
||||
bypassCSP: true,
|
||||
})
|
||||
|
||||
context.setDefaultTimeout(config.browser.timeout || 30000)
|
||||
const page = await context.newPage()
|
||||
|
||||
// Navigate to Twitter login page by default
|
||||
await page.goto('https://x.com/login')
|
||||
|
||||
logger.main.log('Browser initialized')
|
||||
return { browser, context, page }
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize Twitter service and login
|
||||
*/
|
||||
async function initTwitterService(page: Page, context: BrowserContext, _config: Config): Promise<TwitterService> {
|
||||
const authService = new TwitterAuthService(page, context)
|
||||
const timelineService = new TwitterTimelineService(page)
|
||||
const twitterService = new TwitterService(authService, timelineService)
|
||||
|
||||
// Check if we have a saved session
|
||||
try {
|
||||
const sessionSuccess = await authService.checkExistingSession()
|
||||
if (sessionSuccess) {
|
||||
logger.main.log('Successfully loaded existing Twitter session')
|
||||
}
|
||||
else {
|
||||
// Instead of automatic login, navigate to login page
|
||||
logger.main.log('No valid session found, navigating to login page for manual login')
|
||||
await page.goto('https://x.com/login')
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
logger.main.withError(error as Error).warn('Error checking session, navigating to login page')
|
||||
await page.goto('https://x.com/login')
|
||||
}
|
||||
|
||||
// Start session monitoring to automatically save session when user logs in
|
||||
twitterService.startSessionMonitor()
|
||||
logger.main.log('Started automatic session monitoring')
|
||||
|
||||
return twitterService
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize adapters
|
||||
*/
|
||||
async function initAdapters(twitterService: TwitterService, config: Config): Promise<{ airi?: AiriAdapter, mcp?: MCPAdapter }> {
|
||||
const adapters: { airi?: AiriAdapter, mcp?: MCPAdapter } = {}
|
||||
|
||||
// if (config.adapters.airi?.enabled) {
|
||||
// logger.main.log('Starting Airi adapter...')
|
||||
// const { AiriAdapter } = await import('./adapters/airi-adapter')
|
||||
|
||||
// adapters.airi = new AiriAdapter(twitterService, {
|
||||
// url: config.adapters.airi.url,
|
||||
// token: config.adapters.airi.token,
|
||||
// credentials: {},
|
||||
// })
|
||||
|
||||
// await adapters.airi.start()
|
||||
// logger.main.log('Airi adapter started')
|
||||
// }
|
||||
|
||||
if (config.adapters.mcp?.enabled) {
|
||||
logger.main.log('Starting MCP adapter...')
|
||||
const { MCPAdapter } = await import('./adapters/mcp-adapter')
|
||||
|
||||
adapters.mcp = new MCPAdapter(
|
||||
twitterService,
|
||||
config.adapters.mcp.port,
|
||||
)
|
||||
|
||||
await adapters.mcp.start()
|
||||
logger.main.log('MCP adapter started')
|
||||
}
|
||||
|
||||
return adapters
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean up resources
|
||||
*/
|
||||
async function cleanup(
|
||||
adapters: { airi?: AiriAdapter, mcp?: MCPAdapter },
|
||||
context?: BrowserContext,
|
||||
browser?: Browser,
|
||||
) {
|
||||
logger.main.log('Stopping Twitter service...')
|
||||
|
||||
if (adapters.mcp) {
|
||||
await adapters.mcp.stop()
|
||||
logger.main.log('MCP adapter stopped')
|
||||
}
|
||||
|
||||
if (context) {
|
||||
await context.close()
|
||||
}
|
||||
|
||||
if (browser) {
|
||||
await browser.close()
|
||||
logger.main.log('Browser closed')
|
||||
}
|
||||
|
||||
logger.main.log('Twitter service stopped')
|
||||
}
|
||||
|
||||
/**
|
||||
* Set up process shutdown hooks
|
||||
*/
|
||||
function setupShutdownHooks(
|
||||
adapters: { airi?: AiriAdapter, mcp?: MCPAdapter },
|
||||
context?: BrowserContext,
|
||||
browser?: Browser,
|
||||
) {
|
||||
const handleShutdown = async (signal: string) => {
|
||||
logger.main.log(`Received ${signal} signal...`)
|
||||
await cleanup(adapters, context, browser)
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
process.on('SIGINT', () => handleShutdown('exit'))
|
||||
process.on('SIGTERM', () => handleShutdown('termination'))
|
||||
|
||||
process.on('uncaughtException', async (error) => {
|
||||
logger.main.withError(error).error('Uncaught exception')
|
||||
await cleanup(adapters, context, browser)
|
||||
process.exit(1)
|
||||
})
|
||||
}
|
||||
|
||||
// Start application
|
||||
async function bootstrap() {
|
||||
// Initialize logging system
|
||||
initLogger()
|
||||
|
||||
try {
|
||||
const config = useConfigManager().getConfig()
|
||||
logger.main.log('Starting Twitter service...')
|
||||
|
||||
// Initialize core components
|
||||
const { browser, context, page } = await initBrowser(config)
|
||||
const twitterService = await initTwitterService(page, context, config)
|
||||
const adapters = await initAdapters(twitterService, config)
|
||||
|
||||
// Set up shutdown hooks
|
||||
setupShutdownHooks(adapters, context, browser)
|
||||
|
||||
logger.main.log('Twitter service successfully started!')
|
||||
}
|
||||
catch (error) {
|
||||
logger.main.withError(error).error('Startup failed')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
// Handle unhandled rejections
|
||||
process.on('unhandledRejection', (reason) => {
|
||||
logger.main.withError(reason).error('Unhandled Promise rejection:')
|
||||
})
|
||||
}
|
||||
|
||||
bootstrap()
|
||||
@@ -0,0 +1,293 @@
|
||||
import type { Page } from 'playwright'
|
||||
import type { UserLink, UserProfile, UserStats } from '../types/twitter'
|
||||
|
||||
import { logger } from '../utils/logger'
|
||||
import { SELECTORS } from '../utils/selectors'
|
||||
|
||||
/**
|
||||
* Profile Parser
|
||||
* Extracts user profile information directly from the page DOM using Playwright
|
||||
*/
|
||||
export class ProfileParser {
|
||||
/**
|
||||
* Parse user profile from a Twitter profile page
|
||||
* @param page Playwright page instance
|
||||
* @returns Promise resolving to UserProfile object
|
||||
*/
|
||||
static async parseUserProfile(page: Page): Promise<UserProfile> {
|
||||
try {
|
||||
// Extract basic profile info
|
||||
const displayNameElement = await page.$(SELECTORS.PROFILE.DISPLAY_NAME)
|
||||
const displayName = await displayNameElement?.textContent() || 'Unknown User'
|
||||
|
||||
// Get username from URL or profile elements
|
||||
let username = ''
|
||||
const url = page.url()
|
||||
const urlUsername = this.extractUsernameFromUrl(url)
|
||||
|
||||
if (urlUsername) {
|
||||
username = urlUsername
|
||||
}
|
||||
else {
|
||||
// Try to find username in the DOM
|
||||
const usernameElement = await page.$('[data-testid="UserName"] span:has-text("@")')
|
||||
const usernameText = await usernameElement?.textContent()
|
||||
username = usernameText?.replace('@', '') || 'unknown'
|
||||
}
|
||||
|
||||
// Get bio
|
||||
const bioElement = await page.$(SELECTORS.PROFILE.BIO)
|
||||
const bio = await bioElement?.textContent()
|
||||
|
||||
// Get profile images
|
||||
const avatarUrl = await this.extractAvatarUrl(page)
|
||||
const bannerUrl = await this.extractBannerUrl(page)
|
||||
|
||||
// Get statistics
|
||||
const stats = await this.extractUserStats(page)
|
||||
|
||||
// Get join date
|
||||
const joinDate = await this.extractJoinDate(page)
|
||||
|
||||
// Get user links
|
||||
// const _links = await this.extractUserLinks(page)
|
||||
|
||||
const profile: UserProfile = {
|
||||
username,
|
||||
displayName,
|
||||
}
|
||||
|
||||
// Add optional fields if they exist
|
||||
if (bio)
|
||||
profile.bio = bio
|
||||
if (avatarUrl)
|
||||
profile.avatarUrl = avatarUrl
|
||||
if (bannerUrl)
|
||||
profile.bannerUrl = bannerUrl
|
||||
if (stats.followers)
|
||||
profile.followersCount = stats.followers
|
||||
if (stats.following)
|
||||
profile.followingCount = stats.following
|
||||
if (stats.tweets)
|
||||
profile.tweetCount = stats.tweets
|
||||
if (joinDate)
|
||||
profile.joinDate = joinDate
|
||||
|
||||
// Check for verification badge
|
||||
const isVerified = await page.$('[data-testid="icon-verified"]') !== null
|
||||
if (isVerified)
|
||||
profile.isVerified = true
|
||||
|
||||
return profile
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error parsing user profile:', (error as Error).message)
|
||||
|
||||
// Return minimal profile to avoid breaking
|
||||
return {
|
||||
username: 'unknown',
|
||||
displayName: 'Unknown User',
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract username from Twitter profile URL
|
||||
* @param url Twitter profile URL
|
||||
* @returns Username or null if not found
|
||||
*/
|
||||
private static extractUsernameFromUrl(url: string): string | null {
|
||||
try {
|
||||
const match = url.match(/twitter\.com\/([^/]+)/)
|
||||
if (match && match[1] && !['home', 'explore', 'notifications', 'messages'].includes(match[1])) {
|
||||
return match[1]
|
||||
}
|
||||
return null
|
||||
}
|
||||
catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract user statistics (followers, following, tweets)
|
||||
* @param page Playwright page instance
|
||||
* @returns Promise resolving to UserStats object
|
||||
*/
|
||||
private static async extractUserStats(page: Page): Promise<UserStats> {
|
||||
const stats: UserStats = {
|
||||
followers: 0,
|
||||
following: 0,
|
||||
tweets: 0,
|
||||
}
|
||||
|
||||
try {
|
||||
// Get stats container
|
||||
const statsContainer = await page.$(SELECTORS.PROFILE.STATS)
|
||||
if (!statsContainer)
|
||||
return stats
|
||||
|
||||
// Get all stat items
|
||||
const statItems = await statsContainer.$$('a')
|
||||
|
||||
for (const statItem of statItems) {
|
||||
const text = await statItem.textContent() || ''
|
||||
|
||||
if (text.includes('Following')) {
|
||||
const countText = text.replace(/Following.*/, '').trim()
|
||||
stats.following = this.parseStatNumber(countText)
|
||||
}
|
||||
else if (text.includes('Followers')) {
|
||||
const countText = text.replace(/Followers.*/, '').trim()
|
||||
stats.followers = this.parseStatNumber(countText)
|
||||
}
|
||||
else if (text.includes('posts') || text.includes('Posts')) {
|
||||
const countText = text.replace(/posts|Posts.*/, '').trim()
|
||||
stats.tweets = this.parseStatNumber(countText)
|
||||
}
|
||||
}
|
||||
|
||||
return stats
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error extracting user stats:', (error as Error).message)
|
||||
return stats
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract profile avatar URL
|
||||
* @param page Playwright page instance
|
||||
* @returns Promise resolving to avatar URL or undefined
|
||||
*/
|
||||
private static async extractAvatarUrl(page: Page): Promise<string | undefined> {
|
||||
try {
|
||||
const avatarElement = await page.$('img[src*="profile_images"]')
|
||||
const src = await avatarElement?.getAttribute('src')
|
||||
return src || undefined
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error extracting avatar URL:', (error as Error).message)
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract profile banner URL
|
||||
* @param page Playwright page instance
|
||||
* @returns Promise resolving to banner URL or undefined
|
||||
*/
|
||||
private static async extractBannerUrl(page: Page): Promise<string | undefined> {
|
||||
try {
|
||||
const bannerElement = await page.$('img[src*="profile_banners"]')
|
||||
const src = await bannerElement?.getAttribute('src')
|
||||
return src || undefined
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error extracting banner URL:', (error as Error).message)
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract join date from profile
|
||||
* @param page Playwright page instance
|
||||
* @returns Promise resolving to join date string or undefined
|
||||
*/
|
||||
private static async extractJoinDate(page: Page): Promise<string | undefined> {
|
||||
try {
|
||||
// Try to find join date text that usually appears as "Joined Month Year"
|
||||
const joinedText = await page.$('span:has-text("Joined")')
|
||||
if (!joinedText)
|
||||
return undefined
|
||||
|
||||
const fullText = await joinedText.textContent()
|
||||
if (fullText && fullText.includes('Joined')) {
|
||||
// Extract just the date part
|
||||
const datePart = fullText.replace('Joined', '').trim()
|
||||
return datePart || undefined
|
||||
}
|
||||
|
||||
return undefined
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error extracting join date:', (error as Error).message)
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract user links (website, location)
|
||||
* @param page Playwright page instance
|
||||
* @returns Promise resolving to array of user links
|
||||
*/
|
||||
private static async extractUserLinks(page: Page): Promise<UserLink[]> {
|
||||
const links: UserLink[] = []
|
||||
|
||||
try {
|
||||
// Find all link elements in profile
|
||||
const linkElements = await page.$$('a[href^="https"]:not([href*="twitter.com"])')
|
||||
|
||||
for (const linkElement of linkElements) {
|
||||
// Extract href and title
|
||||
const href = await linkElement.getAttribute('href')
|
||||
const title = await linkElement.textContent()
|
||||
|
||||
if (href && title) {
|
||||
links.push({
|
||||
type: 'url',
|
||||
url: href,
|
||||
title,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Try to find location
|
||||
const locationElement = await page.$('span:has-text("Location")')
|
||||
if (locationElement) {
|
||||
const locationText = await locationElement.textContent()
|
||||
if (locationText) {
|
||||
links.push({
|
||||
type: 'location',
|
||||
url: '',
|
||||
title: locationText.replace('Location', '').trim(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
return links
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error extracting user links:', (error as Error).message)
|
||||
return links
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse stat number with K, M suffixes
|
||||
* @param text Number text (e.g., "10.5K")
|
||||
* @returns Parsed number
|
||||
*/
|
||||
private static parseStatNumber(text: string): number {
|
||||
try {
|
||||
text = text.trim()
|
||||
|
||||
if (!text)
|
||||
return 0
|
||||
|
||||
if (text.includes('K')) {
|
||||
return Math.round(Number.parseFloat(text.replace('K', '')) * 1000)
|
||||
}
|
||||
else if (text.includes('M')) {
|
||||
return Math.round(Number.parseFloat(text.replace('M', '')) * 1000000)
|
||||
}
|
||||
|
||||
// Handle other formats like 1,234
|
||||
const normalized = text.replace(/,/g, '')
|
||||
return Number.parseInt(normalized, 10) || 0
|
||||
}
|
||||
catch {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,260 @@
|
||||
import type { ElementHandle, Page } from 'playwright'
|
||||
import type { Tweet } from '../types/twitter'
|
||||
|
||||
import { logger } from '../utils/logger'
|
||||
import { SELECTORS } from '../utils/selectors'
|
||||
|
||||
/**
|
||||
* Tweet Parser
|
||||
* Extracts tweet information directly from the page DOM using Playwright
|
||||
*/
|
||||
export class TweetParser {
|
||||
/**
|
||||
* Parse timeline tweets directly from the page
|
||||
* @param page Playwright page instance
|
||||
* @returns Promise resolving to Tweet array
|
||||
*/
|
||||
static async parseTimelineTweets(page: Page): Promise<Tweet[]> {
|
||||
try {
|
||||
const tweetElements = await page.$$(SELECTORS.TIMELINE.TWEET)
|
||||
logger.parser.log(`Found ${tweetElements.length} tweet elements`)
|
||||
|
||||
const tweets: Tweet[] = []
|
||||
|
||||
for (const tweetElement of tweetElements) {
|
||||
const tweet = await this.extractTweetData(page, tweetElement)
|
||||
if (tweet) {
|
||||
tweets.push(tweet)
|
||||
}
|
||||
}
|
||||
|
||||
return tweets
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error parsing timeline tweets:', (error as Error).message)
|
||||
return []
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract tweet data from tweet element
|
||||
* @param page Playwright page instance
|
||||
* @param tweetElement Tweet element handle
|
||||
* @returns Promise resolving to Tweet object
|
||||
*/
|
||||
static async extractTweetData(page: Page, tweetElement: ElementHandle): Promise<Tweet | null> {
|
||||
try {
|
||||
// Extract tweet ID
|
||||
const id = await this.extractTweetId(tweetElement)
|
||||
|
||||
// Extract tweet text
|
||||
const textElement = await tweetElement.$(SELECTORS.TIMELINE.TWEET_TEXT)
|
||||
const text = textElement ? await textElement.textContent() : ''
|
||||
|
||||
// Extract author info
|
||||
const author = await this.extractAuthorInfo(tweetElement)
|
||||
|
||||
// Extract timestamp
|
||||
const timeElement = await tweetElement.$('time')
|
||||
const timestamp = timeElement ? await timeElement.getAttribute('datetime') : new Date().toISOString()
|
||||
|
||||
// Extract engagement stats
|
||||
const stats = await this.extractTweetStats(tweetElement)
|
||||
|
||||
// Extract media URLs
|
||||
const mediaUrls = await this.extractMediaUrls(tweetElement)
|
||||
|
||||
const tweet: Tweet = {
|
||||
id,
|
||||
text: text || '',
|
||||
author,
|
||||
timestamp: timestamp || new Date().toISOString(),
|
||||
...stats,
|
||||
}
|
||||
|
||||
if (mediaUrls.length > 0) {
|
||||
tweet.mediaUrls = mediaUrls
|
||||
}
|
||||
|
||||
return tweet
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error extracting tweet data:', (error as Error).message)
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract tweet ID from tweet element
|
||||
* @param tweetElement Tweet element handle
|
||||
* @returns Promise resolving to tweet ID
|
||||
*/
|
||||
private static async extractTweetId(tweetElement: ElementHandle): Promise<string> {
|
||||
try {
|
||||
// Try to get ID from status link
|
||||
const statusLink = await tweetElement.$('a[href*="/status/"]')
|
||||
if (statusLink) {
|
||||
const href = await statusLink.getAttribute('href')
|
||||
if (href) {
|
||||
const match = href.match(/\/status\/(\d+)/)
|
||||
if (match && match[1]) {
|
||||
return match[1]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback to a random ID
|
||||
return `tweet-${Date.now()}-${Math.floor(Math.random() * 1000)}`
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error extracting tweet ID:', (error as Error).message)
|
||||
return `tweet-${Date.now()}`
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract author info from tweet element
|
||||
* @param tweetElement Tweet element handle
|
||||
* @returns Promise resolving to author object
|
||||
*/
|
||||
private static async extractAuthorInfo(tweetElement: ElementHandle): Promise<Tweet['author']> {
|
||||
try {
|
||||
// Find author element
|
||||
const authorElement = await tweetElement.$('[data-testid="User-Name"]')
|
||||
if (!authorElement) {
|
||||
return {
|
||||
username: 'unknown',
|
||||
displayName: 'Unknown User',
|
||||
}
|
||||
}
|
||||
|
||||
// Get display name
|
||||
const displayNameElement = await authorElement.$('span:first-child')
|
||||
const displayName = displayNameElement ? await displayNameElement.textContent() || 'Unknown User' : 'Unknown User'
|
||||
|
||||
// Get username
|
||||
const usernameElement = await authorElement.$('span a[href^="/"]')
|
||||
let username = usernameElement ? await usernameElement.textContent() : 'unknown'
|
||||
username = username?.replace('@', '') || 'unknown'
|
||||
|
||||
// Get avatar URL
|
||||
const avatarElement = await tweetElement.$('img[src*="/profile_images/"]')
|
||||
const avatarUrl = avatarElement ? await avatarElement.getAttribute('src') : undefined
|
||||
|
||||
return {
|
||||
username,
|
||||
displayName,
|
||||
...(avatarUrl && { avatarUrl }),
|
||||
}
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error extracting author info:', (error as Error).message)
|
||||
return {
|
||||
username: 'unknown',
|
||||
displayName: 'Unknown User',
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract tweet stats (likes, retweets, replies)
|
||||
* @param tweetElement Tweet element handle
|
||||
* @returns Promise resolving to stats object
|
||||
*/
|
||||
private static async extractTweetStats(tweetElement: ElementHandle): Promise<{
|
||||
likeCount?: number
|
||||
retweetCount?: number
|
||||
replyCount?: number
|
||||
}> {
|
||||
const stats: {
|
||||
likeCount?: number
|
||||
retweetCount?: number
|
||||
replyCount?: number
|
||||
} = {}
|
||||
|
||||
try {
|
||||
// Extract like count
|
||||
const likeElement = await tweetElement.$(SELECTORS.TIMELINE.LIKE_BUTTON)
|
||||
if (likeElement) {
|
||||
const likeCountElement = await likeElement.$('span span')
|
||||
const likeCountText = likeCountElement ? await likeCountElement.textContent() : null
|
||||
stats.likeCount = this.parseCount(likeCountText)
|
||||
}
|
||||
|
||||
// Extract retweet count
|
||||
const retweetElement = await tweetElement.$(SELECTORS.TIMELINE.RETWEET_BUTTON)
|
||||
if (retweetElement) {
|
||||
const retweetCountElement = await retweetElement.$('span span')
|
||||
const retweetCountText = retweetCountElement ? await retweetCountElement.textContent() : null
|
||||
stats.retweetCount = this.parseCount(retweetCountText)
|
||||
}
|
||||
|
||||
// Extract reply count
|
||||
const replyElement = await tweetElement.$(SELECTORS.TIMELINE.REPLY_BUTTON)
|
||||
if (replyElement) {
|
||||
const replyCountElement = await replyElement.$('span span')
|
||||
const replyCountText = replyCountElement ? await replyCountElement.textContent() : null
|
||||
stats.replyCount = this.parseCount(replyCountText)
|
||||
}
|
||||
|
||||
return stats
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error extracting tweet stats:', (error as Error).message)
|
||||
return stats
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract media URLs from tweet element
|
||||
* @param tweetElement Tweet element handle
|
||||
* @returns Promise resolving to array of media URLs
|
||||
*/
|
||||
private static async extractMediaUrls(tweetElement: ElementHandle): Promise<string[]> {
|
||||
try {
|
||||
const mediaElements = await tweetElement.$$('img[src*="pbs.twimg.com/media/"]')
|
||||
|
||||
const mediaUrls: string[] = []
|
||||
for (const mediaElement of mediaElements) {
|
||||
const src = await mediaElement.getAttribute('src')
|
||||
if (src) {
|
||||
mediaUrls.push(src)
|
||||
}
|
||||
}
|
||||
|
||||
return mediaUrls
|
||||
}
|
||||
catch (error) {
|
||||
logger.parser.error('Error extracting media URLs:', (error as Error).message)
|
||||
return []
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse count text (handles K, M suffixes)
|
||||
* @param countText Count text from tweet
|
||||
* @returns Parsed number or undefined
|
||||
*/
|
||||
private static parseCount(countText: string | null): number | undefined {
|
||||
if (!countText)
|
||||
return undefined
|
||||
|
||||
try {
|
||||
countText = countText.trim()
|
||||
if (!countText)
|
||||
return undefined
|
||||
|
||||
if (countText.includes('K')) {
|
||||
return Math.round(Number.parseFloat(countText.replace('K', '')) * 1000)
|
||||
}
|
||||
else if (countText.includes('M')) {
|
||||
return Math.round(Number.parseFloat(countText.replace('M', '')) * 1000000)
|
||||
}
|
||||
|
||||
return Number.parseInt(countText, 10) || undefined
|
||||
}
|
||||
catch {
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
/**
|
||||
* Tweet Interface
|
||||
*/
|
||||
export interface Tweet {
|
||||
id: string
|
||||
text: string
|
||||
author: {
|
||||
username: string
|
||||
displayName: string
|
||||
avatarUrl?: string
|
||||
}
|
||||
timestamp: string
|
||||
likeCount?: number
|
||||
retweetCount?: number
|
||||
replyCount?: number
|
||||
mediaUrls?: string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Tweet Detail
|
||||
*/
|
||||
export interface TweetDetail extends Tweet {
|
||||
replies?: Tweet[]
|
||||
quotedTweet?: Tweet
|
||||
}
|
||||
|
||||
/**
|
||||
* User Profile
|
||||
*/
|
||||
export interface UserProfile {
|
||||
username: string
|
||||
displayName: string
|
||||
bio?: string
|
||||
avatarUrl?: string
|
||||
bannerUrl?: string
|
||||
followersCount?: number
|
||||
followingCount?: number
|
||||
tweetCount?: number
|
||||
isVerified?: boolean
|
||||
joinDate?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Timeline Options
|
||||
*/
|
||||
export interface TimelineOptions {
|
||||
count?: number
|
||||
includeReplies?: boolean
|
||||
includeRetweets?: boolean
|
||||
limit?: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Search Options
|
||||
*/
|
||||
export interface SearchOptions {
|
||||
count?: number
|
||||
filter?: 'latest' | 'photos' | 'videos' | 'top'
|
||||
}
|
||||
|
||||
/**
|
||||
* Post Options
|
||||
*/
|
||||
export interface PostOptions {
|
||||
media?: string[]
|
||||
inReplyTo?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* User Stats
|
||||
*/
|
||||
export interface UserStats {
|
||||
tweets: number
|
||||
following: number
|
||||
followers: number
|
||||
}
|
||||
|
||||
/**
|
||||
* User Link
|
||||
*/
|
||||
export interface UserLink {
|
||||
type: string
|
||||
url: string
|
||||
title: string
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
/**
|
||||
* Safely extract error message from any error type
|
||||
* Handles Error objects, strings, objects, and other types
|
||||
*
|
||||
* @param error - Any error object
|
||||
* @param fallbackMessage - Fallback message when unable to extract a message
|
||||
* @returns Formatted error message
|
||||
*/
|
||||
export function errorToMessage(error: unknown, fallbackMessage = 'Unknown error'): string {
|
||||
if (error === null || error === undefined) {
|
||||
return fallbackMessage
|
||||
}
|
||||
|
||||
// Handle standard Error objects
|
||||
if (error instanceof Error) {
|
||||
return error.message
|
||||
}
|
||||
|
||||
// Handle string errors
|
||||
if (typeof error === 'string') {
|
||||
return error
|
||||
}
|
||||
|
||||
// Handle objects with message property
|
||||
if (typeof error === 'object') {
|
||||
// Check if it has a message property
|
||||
if ('message' in error && typeof (error as any).message === 'string') {
|
||||
return (error as any).message
|
||||
}
|
||||
|
||||
// Try to convert object to string
|
||||
try {
|
||||
return JSON.stringify(error)
|
||||
}
|
||||
catch {
|
||||
// If serialization fails, return object's string representation
|
||||
return String(error)
|
||||
}
|
||||
}
|
||||
|
||||
// For other cases, try to force convert to string
|
||||
return String(error)
|
||||
}
|
||||
|
||||
/**
|
||||
* Create an error with detailed context information
|
||||
*
|
||||
* @param message - Error message
|
||||
* @param originalError - Original error object (optional)
|
||||
* @param context - Additional context information (optional)
|
||||
* @returns Enhanced error object
|
||||
*/
|
||||
export function createError(
|
||||
message: string,
|
||||
originalError?: unknown,
|
||||
context?: Record<string, unknown>,
|
||||
): Error {
|
||||
let errorMessage = message
|
||||
|
||||
// Add original error information
|
||||
if (originalError) {
|
||||
errorMessage += `: ${errorToMessage(originalError)}`
|
||||
}
|
||||
|
||||
// Create new error object
|
||||
const error = new Error(errorMessage)
|
||||
|
||||
// Add context information
|
||||
if (context) {
|
||||
Object.assign(error, { context })
|
||||
}
|
||||
|
||||
return error
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
import path from 'node:path'
|
||||
import { createLogg, Format, LogLevel, setGlobalFormat, setGlobalLogLevel } from '@guiiai/logg'
|
||||
|
||||
import { useConfigManager } from '../config'
|
||||
|
||||
// Track initialization status
|
||||
let isInitialized = false
|
||||
|
||||
// Initialize global logging configuration
|
||||
export function initLogger(): void {
|
||||
if (isInitialized) {
|
||||
return // Prevent multiple initializations
|
||||
}
|
||||
|
||||
// Set global log level
|
||||
setGlobalLogLevel(LogLevel.Debug)
|
||||
setGlobalFormat(Format.Pretty)
|
||||
|
||||
const config = useConfigManager().getConfig()
|
||||
|
||||
const logLevelMap: Record<string, LogLevel> = {
|
||||
error: LogLevel.Error,
|
||||
warn: LogLevel.Warning,
|
||||
info: LogLevel.Log,
|
||||
verbose: LogLevel.Verbose,
|
||||
debug: LogLevel.Debug,
|
||||
}
|
||||
|
||||
setGlobalLogLevel(logLevelMap[config.system?.logLevel] || LogLevel.Debug)
|
||||
|
||||
// Set format based on configuration
|
||||
if (config.system?.logFormat === 'pretty') {
|
||||
setGlobalFormat(Format.Pretty)
|
||||
}
|
||||
else {
|
||||
setGlobalFormat(Format.JSON)
|
||||
}
|
||||
|
||||
isInitialized = true
|
||||
}
|
||||
|
||||
/**
|
||||
* Get logger instance with directory name and filename
|
||||
* @returns logger instance configured with "directoryName/filename"
|
||||
*/
|
||||
export function useLogger(name?: string): ReturnType<typeof createLogg> {
|
||||
if (name)
|
||||
return createLogg(name).useGlobalConfig()
|
||||
|
||||
const stack = new Error('logger').stack
|
||||
const caller = stack?.split('\n')[2]
|
||||
|
||||
// Extract directory, filename and line number from stack trace
|
||||
const match = caller?.match(/(?:([^/]+)\/)?([^/\s]+?)(?:\.[jt]s)?:(\d+)(?::\d+)?\)?$/)
|
||||
const dirName = match?.[1] || path.basename(path.dirname(__filename))
|
||||
const fileName = match?.[2] || path.basename(__filename, '.ts')
|
||||
const lineNumber = match?.[3] || '?'
|
||||
|
||||
return createLogg(`${dirName}/${fileName}:${lineNumber}`).useGlobalConfig()
|
||||
}
|
||||
|
||||
// Create pre-configured loggers for various services
|
||||
export const logger = {
|
||||
auth: useLogger('auth-service'),
|
||||
timeline: useLogger('timeline-service'),
|
||||
browser: useLogger('browser-adapter'),
|
||||
airi: useLogger('airi-adapter'),
|
||||
mcp: useLogger('mcp-adapter'),
|
||||
parser: useLogger('parser'),
|
||||
main: useLogger('twitter-service'),
|
||||
config: useLogger('config'),
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
/**
|
||||
* Request rate limiter
|
||||
* Controls request frequency to Twitter to avoid triggering limits
|
||||
*/
|
||||
export class RateLimiter {
|
||||
private requestHistory: number[] = []
|
||||
private maxRequests: number
|
||||
private timeWindow: number
|
||||
|
||||
/**
|
||||
* Create rate limiter
|
||||
* @param maxRequests Maximum requests within time window
|
||||
* @param timeWindow Time window size (milliseconds)
|
||||
*/
|
||||
constructor(maxRequests: number = 20, timeWindow: number = 60000) {
|
||||
this.maxRequests = maxRequests
|
||||
this.timeWindow = timeWindow
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if request can be sent
|
||||
*/
|
||||
canRequest(): boolean {
|
||||
this.cleanOldRequests()
|
||||
return this.requestHistory.length < this.maxRequests
|
||||
}
|
||||
|
||||
/**
|
||||
* Record a request
|
||||
*/
|
||||
recordRequest(): void {
|
||||
this.requestHistory.push(Date.now())
|
||||
}
|
||||
|
||||
/**
|
||||
* Get wait time until next available request (milliseconds)
|
||||
* Returns 0 if request can be sent now
|
||||
*/
|
||||
getWaitTime(): number {
|
||||
if (this.canRequest()) {
|
||||
return 0
|
||||
}
|
||||
|
||||
const oldestRequest = this.requestHistory[0]
|
||||
return oldestRequest + this.timeWindow - Date.now()
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean expired request records
|
||||
*/
|
||||
private cleanOldRequests(): void {
|
||||
const now = Date.now()
|
||||
const cutoff = now - this.timeWindow
|
||||
this.requestHistory = this.requestHistory.filter(time => time >= cutoff)
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait until request can be sent
|
||||
*/
|
||||
async waitUntilReady(): Promise<void> {
|
||||
const waitTime = this.getWaitTime()
|
||||
if (waitTime > 0) {
|
||||
await new Promise(resolve => setTimeout(resolve, waitTime))
|
||||
}
|
||||
this.recordRequest()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
/**
|
||||
* Twitter website CSS selector constants
|
||||
* Used to locate elements on the page
|
||||
*/
|
||||
export const SELECTORS = {
|
||||
LOGIN: {
|
||||
USERNAME_INPUT: 'input[autocomplete="username"]',
|
||||
PASSWORD_INPUT: 'input[type="password"]',
|
||||
NEXT_BUTTON: 'div[role="button"]:has-text("Next")',
|
||||
LOGIN_BUTTON: 'div[role="button"]:has-text("Log in")',
|
||||
NEXT_BUTTON_ALT: '[data-testid="login-next-button"]',
|
||||
LOGIN_BUTTON_ALT: '[data-testid="login-submit-button"]',
|
||||
},
|
||||
HOME: {
|
||||
TIMELINE: '[data-testid="primaryColumn"]',
|
||||
TRENDING: '[data-testid="sidebarColumn"]',
|
||||
TWEET_COMPOSER: '[data-testid="tweetButtonInline"]',
|
||||
},
|
||||
TIMELINE: {
|
||||
TWEET: '[data-testid="tweet"]',
|
||||
TWEET_TEXT: '[data-testid="tweetText"]',
|
||||
TWEET_TIME: 'time',
|
||||
LIKE_BUTTON: '[data-testid="like"]',
|
||||
RETWEET_BUTTON: '[data-testid="retweet"]',
|
||||
REPLY_BUTTON: '[data-testid="reply"]',
|
||||
AUTHOR_LINK: '[data-testid="User-Name"] a',
|
||||
AUTHOR_NAME: '[data-testid="User-Name"] a div span',
|
||||
AUTHOR_USERNAME: '[data-testid="User-Name"] a div div span',
|
||||
TWEET_STATS: '[data-testid="tweet"] [role="group"]',
|
||||
REPLY_COUNT: '[data-testid="reply"] span span',
|
||||
RETWEET_COUNT: '[data-testid="retweet"] span span',
|
||||
LIKE_COUNT: '[data-testid="like"] span span',
|
||||
VIEW_COUNT: '[data-testid="tweet"] [data-testid="app-text-transition-container"] span span',
|
||||
MEDIA_CONTAINER: '[data-testid="tweetPhoto"]',
|
||||
VIDEO_CONTAINER: '[data-testid="videoPlayer"]',
|
||||
TWEET_LINK: 'a[aria-label*="posted"]',
|
||||
TWEET_ID_CONTAINER: 'a[href*="/status/"]',
|
||||
},
|
||||
PROFILE: {
|
||||
FOLLOW_BUTTON: '[data-testid="followButton"]',
|
||||
UNFOLLOW_BUTTON: '[data-testid="unfollowButton"]',
|
||||
DISPLAY_NAME: '[data-testid="UserName"] div span',
|
||||
USERNAME: '[data-testid="UserName"] div span:has-text("@")',
|
||||
BIO: '[data-testid="UserDescription"]',
|
||||
STATS: '[data-testid="UserProfileStats"]',
|
||||
FOLLOWING_STAT: '[href$="/following"]',
|
||||
FOLLOWERS_STAT: '[href$="/followers"]',
|
||||
AVATAR: '[data-testid="UserAvatar-Container"] img',
|
||||
BANNER: '[data-testid="UserProfileHeader_Items"] img',
|
||||
JOIN_DATE: '[data-testid="UserProfileHeader_Items"] span:has-text("Joined")',
|
||||
LOCATION: '[data-testid="UserProfileHeader_Items"] span:has-text("Location")',
|
||||
WEBSITE: '[data-testid="UserProfileHeader_Items"] a[href^="https"]',
|
||||
VERIFIED_BADGE: '[data-testid="icon-verified"]',
|
||||
},
|
||||
COMPOSE: {
|
||||
TWEET_INPUT: '[data-testid="tweetTextarea_0"]',
|
||||
TWEET_BUTTON: '[data-testid="tweetButtonInline"]',
|
||||
MEDIA_BUTTON: '[data-testid="imageOrGifButton"]',
|
||||
POLL_BUTTON: '[data-testid="createPollButton"]',
|
||||
EMOJI_BUTTON: '[data-testid="emojiButton"]',
|
||||
SCHEDULE_BUTTON: '[data-testid="scheduleButton"]',
|
||||
},
|
||||
SEARCH: {
|
||||
INPUT: '[data-testid="SearchBox_Search_Input"]',
|
||||
RESULT_TAB: '[role="tablist"] [role="presentation"]',
|
||||
PEOPLE_TAB: '[role="tab"]:has-text("People")',
|
||||
LATEST_TAB: '[role="tab"]:has-text("Latest")',
|
||||
TOP_TAB: '[role="tab"]:has-text("Top")',
|
||||
SEARCH_FILTERS: '[data-testid="searchFiltersButton"]',
|
||||
},
|
||||
NOTIFICATIONS: {
|
||||
ITEM: '[data-testid="cellInnerDiv"]',
|
||||
MENTION: '[data-testid="notification"]',
|
||||
FOLLOW: '[data-testid="notification"]',
|
||||
LIKE: '[data-testid="notification"]',
|
||||
RETWEET: '[data-testid="notification"]',
|
||||
},
|
||||
MESSAGES: {
|
||||
CONVERSATION: '[data-testid="conversation"]',
|
||||
MESSAGE_INPUT: '[data-testid="dmComposerTextInput"]',
|
||||
SEND_BUTTON: '[data-testid="dmComposerSendButton"]',
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ESNext",
|
||||
"lib": [
|
||||
"ESNext"
|
||||
],
|
||||
"moduleDetection": "auto",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"resolveJsonModule": true,
|
||||
"strict": true,
|
||||
"strictNullChecks": true,
|
||||
"noImplicitAny": true,
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true,
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"isolatedModules": true,
|
||||
"verbatimModuleSyntax": true,
|
||||
"skipLibCheck": true
|
||||
},
|
||||
"include": [
|
||||
"src/**/*.ts"
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user