# Jobs Scraper Monorepo

A modern monorepo architecture for job scraping and analysis services.

## 🏗️ Architecture

```
jobs-scraper/
├── go.mod                    # Root go.mod with module name
├── Makefile                  # Common build/test tasks
├── .github/workflows/        # CI/CD pipelines
├── tools/                    # Build tools and utilities
├── internal/                 # Private shared packages
│   ├── pkg/
│   │   ├── domain/          # Core domain models
│   │   ├── infrastructure/  # Database, RabbitMQ, HTTP
│   │   └── utils/           # Common utilities
│   └── migrations/          # Database migrations
├── libs/                     # Shared libraries
│   ├── ports/               # Command interfaces
│   ├── repo/                # Data repositories
│   └── server/              # Server utilities
├── services/                 # Deployable services
│   ├── api/                 # HTTP API service
│   ├── scraper-linkedin/    # LinkedIn scraper service
│   └── scraper-glassdoor/   # Glassdoor scraper service
├── apps/                     # Client applications
│   ├── cron-analyzer/       # Job analysis cron
│   └── cv-analyzer/         # CV analysis utility
└── scripts/                  # Deployment and utility scripts
```

## 🚀 Services

### API Service (`services/api/`)
- **Purpose**: HTTP API server with CQRS pattern
- **Tech Stack**: Go, Gorilla Mux, PostgreSQL, RabbitMQ
- **Features**: Job management, Swagger documentation, CORS support

### LinkedIn Scraper (`services/scraper-linkedin/`)
- **Purpose**: Scrapes job postings from LinkedIn
- **Tech Stack**: Go, Playwright
- **Features**: Automated job scraping, RabbitMQ integration

### Glassdoor Scraper (`services/scraper-glassdoor/`)
- **Purpose**: Scrapes job postings from Glassdoor
- **Tech Stack**: TypeScript, Node.js
- **Features**: Web scraping, data processing

## 📱 Applications

### Cron Analyzer (`apps/cron-analyzer/`)
- **Purpose**: Analyzes jobs with AI on a schedule
- **Tech Stack**: Go, OpenRouter API
- **Features**: Job analysis, status updates

### CV Analyzer (`apps/cv-analyzer/`)
- **Purpose**: Analyzes CVs against job requirements
- **Tech Stack**: Go, OpenRouter API
- **Features**: CV matching, skill analysis

## 🛠️ Development

### Prerequisites
- Go 1.24+
- Node.js 18+
- Docker & Docker Compose
- PostgreSQL 15+
- RabbitMQ 3+

### Quick Start

1. **Clone the repository**
   ```bash
   git clone <repository-url>
   cd jobs-scraper
   ```

2. **Start infrastructure**
   ```bash
   make rabbitmq-server
   # In another terminal:
   docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=password postgres:15
   ```

3. **Run services**
   ```bash
   # API Service
   make run-api

   # LinkedIn Scraper
   make run-scraper-linkedin

   # Glassdoor Scraper
   make run-scraper-glassdoor

   # Cron Analyzer
   make run-cron-analyzer
   ```

### Available Commands

#### Services
- `make run-api` - Run API service
- `make build-api` - Build API service
- `make run-scraper-linkedin` - Run LinkedIn scraper
- `make build-scraper-linkedin` - Build LinkedIn scraper
- `make run-scraper-glassdoor` - Run Glassdoor scraper
- `make build-scraper-glassdoor` - Build Glassdoor scraper

#### Applications
- `make run-cron-analyzer` - Run cron analyzer
- `make build-cron-analyzer` - Build cron analyzer
- `make run-cv-analyzer` - Run CV analyzer
- `make build-cv-analyzer` - Build CV analyzer

#### Infrastructure
- `make rabbitmq-server` - Start RabbitMQ server
- `make swagger` - Generate API documentation
- `make clean` - Clean build artifacts

#### Legacy Aliases (Backward Compatibility)
- `make run` - Alias for `run-api`
- `make scraper` - Alias for `run-scraper-linkedin`
- `make cron` - Alias for `run-cron-analyzer`

## 🐳 Docker

Each service has its own Dockerfile and can be built independently:

```bash
# Build API service
cd services/api
docker build -t jobs-scraper-api .

# Build LinkedIn scraper
cd services/scraper-linkedin
docker build -t jobs-scraper-linkedin .
```

## 🔧 Configuration

Environment variables are managed through `.env` files:

- `.env` - Production configuration
- `.local.env` - Local development configuration

## 🧪 Testing

Run tests for individual services:

```bash
# API Service
cd services/api && go test ./...

# LinkedIn Scraper
cd services/scraper-linkedin && go test ./...

# Cron Analyzer
cd apps/cron-analyzer && go test ./...
```

## 📊 Monitoring

- **API Documentation**: http://localhost:8080/swagger/
- **RabbitMQ Management**: http://localhost:15672 (guest/guest)
- **Health Checks**: Built into each service

## 🚀 Deployment

The monorepo includes GitHub Actions workflows for:
- Automated testing
- Docker image building
- Multi-service deployment

## 🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

## 📄 License

[Add your license information here]

## 🆘 Support

For support and questions, please [create an issue](issues) or contact the development team.