Gamestudio Links
Zorro Links
Newest Posts
Issue with converted data
by steventrading15. 12/12/24 21:30
MT4 Bridge connectivity issue
by FXSurgeon. 12/11/24 14:40
Zorro Gives Wrong Calculation During Backtest
by vicknick. 12/11/24 06:31
Price is not shifted with PriceOffset
by vicknick. 12/10/24 13:30
Zorro Gives Price that Doesn't Exist
by jcl. 12/10/24 08:55
Multicore Optimize
by jcl. 12/10/24 08:48
Get timestamp as a var
by Mac81. 12/08/24 01:50
Multicore Optimizations
by Cristoig2001. 12/05/24 14:14
AUM Magazine
Latest Screens
Galactic Strike X
Zeal-X2
The Bible Game
A psychological thriller game
Who's Online Now
1 registered members (sequi), 648 guests, and 4 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sequi, MikeHe, Algospider, jordi, FXSurgeon
19090 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 5 of 5 1 2 3 4 5
DRL direct Feed from Zorro [Re: TipmyPip] #488431
11/09/24 17:07
11/09/24 17:07
Joined: Sep 2017
Posts: 98
T
TipmyPip Offline OP
Junior Member
TipmyPip  Offline OP
Junior Member
T

Joined: Sep 2017
Posts: 98
It’s possible to pass the live data feed from Zorro Trader directly to a Deep Reinforcement Learning (DRL) model without having to call a function from Zorro each time. This can be achieved by running the DRL model in a separate Python script or process that continuously ingests the live data from Zorro in real-time, allowing it to learn and adapt independently. Here’s how to set this up:

Steps to Integrate Zorro’s Live Feed Directly into a DRL Model
Set Up a Data Stream from Zorro to Python:

Configure Zorro to save live market data into a continuously updated file or to use an inter-process communication (IPC) method, such as sockets or shared memory, to feed data into the Python DRL model.
Run the DRL Model as an Independent Python Process:

Implement the DRL model in a standalone Python script that continuously reads data from the Zorro feed and performs training in real-time.
The DRL model can process the data asynchronously and make trading decisions based on updated observations.
Implement Communication Back to Zorro for Trade Execution:

When the DRL model decides on a trade action, it can communicate this action back to Zorro, which will execute the trade.

Implementation Options
Option 1: Data Streaming Using File I/O
Zorro Script for Writing Data Feed:

Modify the Zorro run function to write the latest market data to a CSV file (e.g., live_data.csv) at each bar interval.

Code
function run()
{
    vars InputData = series(priceClose());  // Example data series
    
    // Save data to CSV file for the DRL model
    file_write("Data/live_data.csv", InputData, 0);

    // Execute trades based on external signals
    var signal = file_read("Data/drl_signal.txt");  // Read DRL action signal
    if (signal > 0.5)
        enterLong();
    else if (signal <= 0.5)
        enterShort();
}


Python DRL Script (Independent Process):

In a separate Python script, the DRL model continuously monitors live_data.csv for new data and updates its learning and action policy accordingly.

Code
import torch
import numpy as np
import time
from model import TradingModel  # Define your model in model.py

model = TradingModel(input_size=8, hidden_size=256)
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
criterion = torch.nn.BCELoss()

def train_model(input_data, target):
    model.train()
    input_tensor = torch.tensor(input_data, dtype=torch.float32)
    target_tensor = torch.tensor([target], dtype=torch.float32)

    optimizer.zero_grad()
    prediction = model(input_tensor)
    loss = criterion(prediction, target_tensor)
    loss.backward()
    optimizer.step()
    return loss.item()

while True:
    # Read the latest data from live_data.csv
    try:
        data = np.loadtxt("Data/live_data.csv", delimiter=',')
        input_data = data[-1]  # Get the most recent row of data
        prediction = model(torch.tensor(input_data, dtype=torch.float32)).item()
        
        # Save the action decision back to a file that Zorro reads
        with open("Data/drl_signal.txt", "w") as f:
            f.write("1" if prediction > 0.5 else "0")  # Save the signal for Zorro
        
        # Optionally train model with feedback (e.g., profit from the last trade)
        # target = get_trade_feedback()  # You may define a function to get trade results
        # train_model(input_data, target)

    except Exception as e:
        print(f"Error reading data or training model: {e}")

    time.sleep(1)  # Polling interval, adjust based on data frequency


Option 2: Using Sockets for Real-Time Data Streaming
Set Up a Socket Server in the Python DRL Script:

Run a socket server in Python to continuously receive data from Zorro and send back trade actions.

Code
import socket
import torch
import numpy as np
from model import TradingModel

model = TradingModel(input_size=8, hidden_size=256)

# Initialize socket server
HOST = 'localhost'  
PORT = 65432        
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((HOST, PORT))
server.listen()

print("Waiting for Zorro connection...")

conn, addr = server.accept()
print(f"Connected by {addr}")

while True:
    data = conn.recv(1024)  # Adjust buffer size as needed
    if not data:
        break
    
    # Convert data to numpy array
    input_data = np.frombuffer(data, dtype=np.float32)
    
    # Make a prediction
    prediction = model(torch.tensor(input_data)).item()

    # Send decision to Zorro
    decision = b'1' if prediction > 0.5 else b'0'
    conn.sendall(decision)


Modify the Zorro Script to Send Data via Socket:

In the Zorro script, connect to the Python socket server to send live data and receive the model’s action signal.

Code
function run()
{
    vars InputData = series(priceClose());
    double Data[NumSignals];
    for(int i = 0; i < NumSignals; i++)
        Data[i] = InputData[i];

    // Send data to Python via socket
    int socket = socket_connect("localhost", 65432);
    if (socket != 0) {
        socket_send(socket, Data, sizeof(Data));  // Send data
        int action = socket_receive(socket);      // Receive action decision
        socket_close(socket);

        // Act on received decision
        if (action == '1')
            enterLong();
        else if (action == '0')
            enterShort();
    }
}


By running the DRL model as an independent process, it can continuously learn and adapt from real-time data while only sending trade decisions back to Zorro for execution. This setup provides the flexibility to train the DRL model in live environments without interrupting Zorro’s main trading operations.

In the Zorro platform, socket-based networking functions like socket_connect, socket_send, and socket_receive are not built-in, as Zorro’s lite-C scripting language does not natively support socket programming. However, you can achieve socket communication in Zorro using DLLs (Dynamic Link Libraries) to bridge between Zorro and Python for real-time data feeds.

Steps to Set Up Socket Communication Between Zorro and Python Using a DLL
Create a C/C++ Socket DLL:

Write a C/C++ library that implements socket functions (connect, send, receive, close).
Compile the library into a DLL that Zorro can load and call directly from the lite-C script.
Use the DLL in Zorro:

Load the DLL in your Zorro script.
Use the DLL functions to send data to and receive data from a Python socket server.
Example of a Simple C/C++ Socket DLL
Here’s a minimal example to create a C++ DLL for socket communication. This example provides functions for connecting, sending, receiving, and closing a socket.

C++ Socket DLL Code
Save this file as SocketDLL.cpp, and then compile it into a DLL.

Code
#include <winsock2.h>
#include <ws2tcpip.h>

#pragma comment(lib, "ws2_32.lib")

extern "C" {
    SOCKET sock = INVALID_SOCKET;

    __declspec(dllexport) int socket_connect(const char* ip, int port) {
        WSADATA wsaData;
        WSAStartup(MAKEWORD(2,2), &wsaData);
        
        sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
        if (sock == INVALID_SOCKET) return -1;

        sockaddr_in clientService;
        clientService.sin_family = AF_INET;
        clientService.sin_addr.s_addr = inet_addr(ip);
        clientService.sin_port = htons(port);

        if (connect(sock, (SOCKADDR*)&clientService, sizeof(clientService)) == SOCKET_ERROR) {
            closesocket(sock);
            WSACleanup();
            return -1;
        }
        return 0;
    }

    __declspec(dllexport) int socket_send(const char* data, int length) {
        if (sock == INVALID_SOCKET) return -1;
        return send(sock, data, length, 0);
    }

    __declspec(dllexport) int socket_receive(char* buffer, int length) {
        if (sock == INVALID_SOCKET) return -1;
        return recv(sock, buffer, length, 0);
    }

    __declspec(dllexport) void socket_close() {
        if (sock != INVALID_SOCKET) {
            closesocket(sock);
            WSACleanup();
        }
    }
}


Explanation:
socket_connect: Connects to the given IP and port.
socket_send: Sends data through the connected socket.
socket_receive: Receives data from the socket into a buffer.
socket_close: Closes the socket and cleans up resources.
Compile the DLL
Use a C++ compiler, like Microsoft Visual Studio, to compile this code into a DLL (e.g., SocketDLL.dll). Ensure it’s a 64-bit DLL if you’re using the 64-bit version of Zorro.

Zorro Script to Use the DLL
Place the compiled DLL (SocketDLL.dll) in the same directory as your Zorro script.
Use the DLL functions in your Zorro script to connect, send, and receive data.
Zorro lite-C Script Example

Code
function run()
{
    // Load the socket DLL
    int connect_result = call("SocketDLL.dll", "socket_connect", "localhost", 65432);

    if (connect_result == 0) {
        printf("\nConnected to socket.");

        // Example data to send
        double data[8] = { 1.2, 2.3, 3.4, 4.5, 5.6, 6.7, 7.8, 8.9 };
        call("SocketDLL.dll", "socket_send", data, sizeof(data));

        // Buffer for receiving data
        char buffer[8];
        int bytes_received = call("SocketDLL.dll", "socket_receive", buffer, sizeof(buffer));
        
        if (bytes_received > 0) {
            printf("\nReceived data: %s", buffer);  // Example printout
        }
        
        // Close the socket connection
        call("SocketDLL.dll", "socket_close");
    } else {
        printf("\nFailed to connect to socket.");
    }
}


Explanation of the Zorro Script
call() Function: This function is used to call functions from the DLL in Zorro.
call("SocketDLL.dll", "socket_connect", "localhost", 65432) connects to the Python server running on localhost at port 65432.
call("SocketDLL.dll", "socket_send", data, sizeof(data)) sends the data array to the Python server.
call("SocketDLL.dll", "socket_receive", buffer, sizeof(buffer)) receives data from the server into the buffer.
call("SocketDLL.dll", "socket_close") closes the socket connection.
Python Socket Server to Receive Data from Zorro

Code
import socket

HOST = 'localhost'
PORT = 65432

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((HOST, PORT))
server.listen()
print("Server listening...")

conn, addr = server.accept()
print(f"Connected by {addr}")

while True:
    data = conn.recv(1024)
    if not data:
        break
    print("Received data:", data)

    # Send response
    response = b"1"  # For example, send a '1' signal for buy
    conn.sendall(response)

conn.close()


One important note for all traders and programmers, As previously noted, we all want final solutions without having any bugs, scripts that are unlogical, and hard-to-understand code, but the whole point of these examples, it to learn to use ZorroGPT and find faster solutions for your imaginations.

Last edited by TipmyPip; 11/10/24 03:17.
Multi Agent DRL with Simple Stategy [Re: TipmyPip] #488432
11/10/24 10:36
11/10/24 10:36
Joined: Sep 2017
Posts: 98
T
TipmyPip Offline OP
Junior Member
TipmyPip  Offline OP
Junior Member
T

Joined: Sep 2017
Posts: 98
To leverage Zorro Trader’s efficient, event-driven structure for managing a multi-agent Deep Reinforcement Learning (DRL) model setup, we can design the system to optimize computational speed and real-time trade execution while leaving the resource-intensive training to Python. Here’s an approach that maximizes Zorro's fast computation for managing multi-agent DRL models.

Key Principles for Zorro’s Role in Multi-Agent Management
Separation of Prediction and Training:

Zorro handles trade execution and prediction requests, which are computationally light and can run at high frequency.
The heavier tasks—training and tuning the DRL models—are handled by Python independently, reducing Zorro’s load and keeping trade execution highly responsive.
Socket Communication for Real-Time Data Transfer:

Zorro communicates with Python via a socket connection, sending real-time data for each agent and receiving predictions.
This eliminates I/O delays from file reads and writes, allowing Zorro to request predictions instantly and act on them immediately.
Dynamic and Concurrent Agent Management:

Zorro uses its fast computational ability to request predictions concurrently across multiple agents and assets.
It can trigger trade decisions for several agents in parallel, enabling highly responsive multi-asset or multi-strategy trading.
Implementation Strategy
Here’s how to set up Zorro to manage multiple agents effectively:

Zorro: Efficient, Concurrent Data Handling and Trade Execution
Python: Dedicated, Asynchronous Training with Real-Time Prediction Server
Step-by-Step Code Structure
1. Python Socket Server Code (Handling Multi-Agent Predictions and Training)
In Python, we set up a socket server that:

Listens for data requests from Zorro.
Uses pre-trained models to make quick predictions.
Runs training processes in the background asynchronously, ensuring it can respond to prediction requests without delay.

Code
import socket
import torch
import numpy as np
from model import TradingModel  # Assume TradingModel is defined appropriately

# Initialize models and load them if available
agents = {
    "EURUSD": TradingModel(input_size=5, hidden_layers=2, neurons_per_layer=64),
    "GBPUSD": TradingModel(input_size=5, hidden_layers=2, neurons_per_layer=64),
    "EURGBP": TradingModel(input_size=5, hidden_layers=2, neurons_per_layer=64)
}

for name, model in agents.items():
    try:
        model.load_state_dict(torch.load(f"{name}_model.pth"))
        model.eval()
    except FileNotFoundError:
        print(f"No saved model found for {name}, starting fresh.")

# Start a socket server to handle prediction requests
HOST = 'localhost'
PORT = 65432
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((HOST, PORT))
server.listen()
print("Waiting for Zorro connection...")

conn, addr = server.accept()
print(f"Connected by {addr}")

while True:
    data = conn.recv(1024)
    if not data:
        break
    
    try:
        # Parse data to identify agent and extract feature inputs
        data_str = data.decode().split(',')
        agent_name = data_str[0]
        input_data = np.array(data_str[1:], dtype=np.float32)
        
        # Make a prediction if the agent exists
        if agent_name in agents:
            model = agents[agent_name]
            input_tensor = torch.tensor(input_data, dtype=torch.float32)
            with torch.no_grad():
                prediction = model(input_tensor).item()
            decision = '1' if prediction > 0.5 else '0'
        else:
            decision = '0'  # Default action if agent is unrecognized

        # Send the prediction decision back to Zorro
        conn.sendall(decision.encode())

    except Exception as e:
        print(f"Error processing request: {e}")
        break

conn.close()



2. Zorro Code with Socket Communication and Concurrent Agent Management
In Zorro, we:

Use a single socket_connect for each agent.
Request predictions concurrently and execute trades immediately after receiving each prediction.

Code
function run()
{
    // Define data feeds for each pair
    double EURUSD_Data[5] = { priceHigh(), priceLow(), priceClose(), priceOpen(), Vol() };
    double GBPUSD_Data[5] = { priceHigh(), priceLow(), priceClose(), priceOpen(), Vol() };
    double EURGBP_Data[5] = { priceHigh(), priceLow(), priceClose(), priceOpen(), Vol() };

    // Connect to Python server
    int socket = socket_connect("localhost", 65432);
    if (socket == 0) {
        printf("\nFailed to connect to Python socket.");
        return;
    }

    // Agent 1: EURUSD - Send data and get prediction
    string eurusd_data_str = strf("EURUSD,%f,%f,%f,%f,%f", EURUSD_Data[0], EURUSD_Data[1], EURUSD_Data[2], EURUSD_Data[3], EURUSD_Data[4]);
    socket_send(socket, eurusd_data_str, strlen(eurusd_data_str));
    char buffer[2];
    int bytes_received = socket_receive(socket, buffer, sizeof(buffer));
    if (bytes_received > 0 && buffer[0] == '1')
        enterLong("EURUSD");
    else
        enterShort("EURUSD");

    // Agent 2: GBPUSD - Send data and get prediction
    string gbpusd_data_str = strf("GBPUSD,%f,%f,%f,%f,%f", GBPUSD_Data[0], GBPUSD_Data[1], GBPUSD_Data[2], GBPUSD_Data[3], GBPUSD_Data[4]);
    socket_send(socket, gbpusd_data_str, strlen(gbpusd_data_str));
    bytes_received = socket_receive(socket, buffer, sizeof(buffer));
    if (bytes_received > 0 && buffer[0] == '1')
        enterLong("GBPUSD");
    else
        enterShort("GBPUSD");

    // Agent 3: EURGBP - Send data and get prediction
    string eurgbp_data_str = strf("EURGBP,%f,%f,%f,%f,%f", EURGBP_Data[0], EURGBP_Data[1], EURGBP_Data[2], EURGBP_Data[3], EURGBP_Data[4]);
    socket_send(socket, eurgbp_data_str, strlen(eurgbp_data_str));
    bytes_received = socket_receive(socket, buffer, sizeof(buffer));
    if (bytes_received > 0 && buffer[0] == '1')
        enterLong("EURGBP");
    else
        enterShort("EURGBP");

    // Close socket connection
    socket_close(socket);
}


Explanation of the Zorro Code
Concurrent Data Handling:

Zorro gathers data for each agent and sends it independently over the socket.
Each agent’s prediction is received immediately, and trades are executed without waiting for other predictions.
Direct Trade Execution:

Based on the prediction received from Python, Zorro immediately triggers a Long or Short position for each currency pair.
This approach leverages Zorro’s fast computation for rapid trade decisions, as the time-consuming training process is handled asynchronously by Python.
Benefits of This Setup
Parallel Prediction Requests:

Each agent’s data is processed individually, allowing predictions to be handled concurrently.
Zorro executes trades immediately, maximizing the benefit of fast computation.
Efficient Use of Resources:

Training is handled asynchronously by Python, while Zorro focuses purely on trade execution and prediction requests.
The socket communication enables real-time data exchange, ensuring Zorro’s high-frequency capabilities are fully utilized.
Scalable to More Agents:

Adding new agents is straightforward: simply expand the data gathering and prediction request handling sections in both Zorro and Python, as each agent operates independently in this setup.
By separating Zorro’s fast execution from Python’s computational load, this setup achieves efficient multi-agent management, ensuring real-time responsiveness and scalability.

Page 5 of 5 1 2 3 4 5

Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1