LiveCheck FREE
Check Program Bugs Anywhere - Track Your Logs On-the-Go
Have you ever been worried about your program (e.g., a deep learning model training process) running on a computer while on a date with your girlfriend?
Features
- Receive immediate notifications on your phone about
bugs
(i.e.,exceptions
) occurring in your code running on your computer, from anywhere and at anytime. - Track program logs (e.g., training
loss
andaccuracy
in machine learning) running on your computer using your phone. - Visualize your logs with
interactive graphs
on your phone. - Easy to use with:
- Python package requiring just a few lines of code to integrate into your projects.
- Mobile app featuring a friendly and simple interface.
Demo Video
Quick Start
Mobile App
- Download the iOS or Android mobile app named
LiveCheck+
from the App Store or Google Play.
- Open the app and sign in with either your Apple, Google, or GitHub account.
- Check the
Your ID
andAccess Key
shown on your phone.
Python Package
- Install the Python package named
livecheck_python
usingpip
.
pip install livecheck_python
- Import and initialize LiveCheck in your code using the
Your ID
andAccess Key
shown on theLiveCheck+
app on your phone. Any bugs (i.e., exceptions) occurring in your code will be notified to your phone with just these two lines.
from livecheck_python import LiveCheck
livecheck = LiveCheck(your_id='', access_key='')
- Save logs to the cloud over time to track and visualize them on your phone.
livecheck.log(value={'loss': loss, 'val_loss': val_loss, 'accuracy': accuracy, 'val_accuracy': val_accuracy})
User Guide
Mobile App
- Run name (actually the timestamp when the run started).
- Project name (optional).
- Tap to view hyperparameters of the run (optional).
- List of logs in the run (when new logs were saved to the cloud, this list is automatically updated).
- Interactive graph which visualizes data in the logs (each line corresponds to one field in the logs).
- Tap to show or hide lines.
- Drag to change window sizes.
- Swipe right or left to view previous or next runs.
- Tap to delete the run.
Python Package
Constructor
LiveCheck (
your_id: str,
access_key: str,
project_name: Optional[str] = '',
hyperparams: Optional[dict] = None,
notification_period: Optional[float] = 0
) -> LiveCheck
your_id | required | string | Your email used to log in to the mobile app |
access_key | required | string | An access key shown in your account on the mobile app |
project_name | optional | string | Set the project name to make it easier to distinguish logs of different projects on the mobile app |
hyperparams | optional | dictionary | Hyperparameters used to train models in the run (e.g., learning rate) |
notification_period | optional | float | The time in seconds between two consecutive notifications pushed to your mobile. A larger number means fewer notifications. |
Set Project Name
set_project_name (
value: str
) -> None
Instead of setting project_name
when initializing LiveCheck using the constructor function above, you can call livecheck.set_project_name()
.
Set Hyperparameters
set_hyperparams (
value: dict
) -> None
Instead of setting hyperparams
when initializing LiveCheck using the constructor function above, you can call livecheck.set_hyperparams()
.
Currently, the following types of hyperparameters are supported: int
, float
, bool
, and str
.
Set Notification Period
set_notification_period (
value: float
) -> None
Instead of setting notification_period
when initializing LiveCheck using the constructor function above, you can call livecheck.set_notification_period()
.
Notify Exception
After initializing a LiveCheck object using the constructor function above, any exceptions occurring in your code will be automatically notified to your mobile without needing to do anything else.
Save Logs
log (
value: dict,
log_id: Optional[int] = None
) -> None
You can call livecheck.log()
after each event (e.g., after each epoch) you want to save logs to check on the mobile app. You can set a custom log_id
for each function call. Otherwise, the log_id
will start from 1 and automatically increase by 1 after each function call. The logs saved to the cloud by calling this function will be notified and updated to the mobile app on your phone. Two consecutive notifications will be at least 30 seconds apart.
Currently, the following types of logs are supported: int
, float
, bool
, and str
.
Notices
- Currently, the following types of logs and hyperparameters are supported:
int
,float
,bool
, andstr
. - The maximum number of runs in one day is 1000.
- The maximum number of logs per run is 1000. If a run has more than 1000 logs, it is automatically separated and saved as a new run.
- The oldest run will be automatically deleted when a new run is saved to the cloud and the number of runs exceeds 10000.
- The minimum time between two
livecheck.log()
function calls is 0.2 seconds. - The minimum time between two consecutive notifications to the mobile app on your phone is 30 seconds.
Examples
Simple Example with Random Data
Do not forget to enter your_id
and access_key
, as shown on your phone, into line 11 of the code below.
import random
import math
import time
from random import choice, randint
from string import ascii_lowercase
# Import LiveCheck class from livecheck_python package
from livecheck_python import LiveCheck
# Initialize LiveCheck with user credentials
livecheck = LiveCheck(your_id=, access_key=)
# Generate a random project name
project_name = 'project_' + ''.join(choice(ascii_lowercase) for _ in range(randint(1, 20)))
livecheck.set_project_name(value=project_name)
# Set hyperparameters
N_EPOCHS = 10
livecheck.set_hyperparams(value={'n_epochs': N_EPOCHS, 'learning_rate': 0.001})
# Set notification period
livecheck.set_notification_period(value=0)
# Function to generate synthetic metrics for each epoch
def generate_metrics(epoch):
metrics = {
'loss': math.exp(-epoch) + random.gauss(0, 0.05),
'val_loss': math.exp(-epoch) + random.gauss(0, 0.07),
'accuracy': 1 / (1 + math.exp(-0.5 * epoch)) + random.gauss(0, 0.02),
'val_accuracy': 1 / (1 + math.exp(-0.5 * epoch)) + random.gauss(0, 0.03)
}
return metrics
# Training loop
for epoch in range(1, N_EPOCHS + 1):
# Log metrics for the current epoch
metrics = generate_metrics(epoch)
livecheck.log(value=metrics)
# Sleep to simulate training time between epochs
if epoch < N_EPOCHS:
time.sleep(40)
Keras
It is required to install tensorflow
using the command pip install tensorflow
. Moreover, do not forget to enter your_id
and access_key
, as shown on your phone, into line 10 of the code below.
# Import LiveCheck class from livecheck_python package
from livecheck_python import LiveCheck
# Import Keras and TensorFlow
from tensorflow.keras import datasets, layers, models
from tensorflow.keras.callbacks import Callback
from tensorflow.keras.optimizers import Adam
# Initialize LiveCheck with user credentials
livecheck = LiveCheck(your_id=, access_key=)
# Set project name
livecheck.set_project_name(value='project_cifar100')
# Set hyperparameters
N_EPOCHS = 50
learning_rate = 0.001
livecheck.set_hyperparams(value={'n_epochs': N_EPOCHS, 'learning_rate': learning_rate})
# Set notification period
livecheck.set_notification_period(value=0)
# Load CIFAR-100 dataset
(train_images, train_labels), (test_images, test_labels) = datasets.cifar100.load_data()
# Normalize the images
train_images, test_images = train_images / 255.0, test_images / 255.0
# Define a simple CNN model
model = models.Sequential([
layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.Flatten(),
layers.Dense(64, activation='relu'),
layers.Dense(100, activation='softmax')
])
# Compile the model
model.compile(optimizer=Adam(learning_rate=learning_rate),
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
# Custom callback to log metrics to LiveCheck
class CustomCallback(Callback):
def on_epoch_end(self, epoch, logs=None):
metrics = {
'loss': logs['loss'],
'val_loss': logs['val_loss'],
'accuracy': logs['accuracy'],
'val_accuracy': logs['val_accuracy']
}
livecheck.log(value=metrics)
# Train the model
model.fit(train_images, train_labels, epochs=N_EPOCHS,
validation_data=(test_images, test_labels),
callbacks=[CustomCallback()])
PyTorch
It is required to install torch
, torchvision
, and numpy
using the command pip install torch==2.3.1 torchvision==0.18.1 numpy==1.26.4
. Moreover, do not forget to enter your_id
and access_key
, as shown on your phone, into line 97 of the code below.
# Import LiveCheck class from livecheck_python package
from livecheck_python import LiveCheck
# Import PyTorch
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import torchvision
import torchvision.transforms as transforms
from torch.utils.data import DataLoader
# Training function
def train_model():
# Loop over the dataset multiple times
for epoch in range(N_EPOCHS):
running_loss = 0.0
correct = 0
total = 0
net.train()
for i, data in enumerate(trainloader, 0):
# Get the inputs; data is a list of [inputs, labels]
inputs, labels = data
# Zero the parameter gradients
optimizer.zero_grad()
# Forward + backward + optimize
outputs = net(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
# Print statistics
running_loss += loss.item()
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
train_loss = running_loss / len(trainloader)
train_accuracy = correct / total
# Validation step
net.eval()
val_loss = 0.0
correct = 0
total = 0
with torch.no_grad():
for data in testloader:
images, labels = data
outputs = net(images)
loss = criterion(outputs, labels)
val_loss += loss.item()
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
val_loss /= len(testloader)
val_accuracy = correct / total
# Log metrics to LiveCheck
metrics = {
'loss': train_loss,
'val_loss': val_loss,
'accuracy': train_accuracy,
'val_accuracy': val_accuracy
}
livecheck.log(value=metrics)
print(f'Epoch {epoch+1}/{N_EPOCHS}, Loss: {train_loss:.4f}, Accuracy: {train_accuracy:.4f}, Val Loss: {val_loss:.4f}, Val Accuracy: {val_accuracy:.4f}')
print('Finished Training')
# Define a simple CNN model
class SimpleCNN(nn.Module):
def __init__(self):
super(SimpleCNN, self).__init__()
self.conv1 = nn.Conv2d(3, 32, 3, padding=1)
self.pool = nn.MaxPool2d(2, 2)
self.conv2 = nn.Conv2d(32, 64, 3, padding=1)
self.conv3 = nn.Conv2d(64, 64, 3, padding=1)
self.fc1 = nn.Linear(64 * 4 * 4, 64)
self.fc2 = nn.Linear(64, 100)
def forward(self, x):
x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))
x = self.pool(F.relu(self.conv3(x)))
x = x.view(-1, 64 * 4 * 4)
x = F.relu(self.fc1(x))
x = self.fc2(x)
return x
if __name__ == '__main__':
# Initialize LiveCheck with user credentials
livecheck = LiveCheck(your_id=, access_key=)
# Set project name
livecheck.set_project_name(value='project_cifar100')
# Set hyperparameters
N_EPOCHS = 50
learning_rate = 0.001
batch_size = 64
livecheck.set_hyperparams(value={'n_epochs': N_EPOCHS, 'learning_rate': learning_rate, 'batch_size': batch_size})
# Set notification period
livecheck.set_notification_period(value=0)
# Load CIFAR-100 dataset
transform = transforms.Compose(
[transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
trainset = torchvision.datasets.CIFAR100(root='./data', train=True,
download=True, transform=transform)
trainloader = DataLoader(trainset, batch_size=batch_size,
shuffle=True, num_workers=2)
testset = torchvision.datasets.CIFAR100(root='./data', train=False,
download=True, transform=transform)
testloader = DataLoader(testset, batch_size=batch_size,
shuffle=False, num_workers=2)
net = SimpleCNN()
# Define loss function and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(net.parameters(), lr=learning_rate)
train_model()
Citation
If you use this for your work, please cite the manuscript mentioned below in the outcomes of your work (e.g., academic papers).
@misc{livecheck,
title={LiveCheck: Check Program Bugs Anywhere - Track Your Logs On-the-Go},
author={},
journal={arXiv},
year={2024}
}
Contact
Please feel free to contact me at tuan.t.d@ieee.org for inquiries or collaboration opportunities.