Installation and first inference
Run real weights on your machine.
- Code, environment, and weights
- Choose CPU or GPU deliberately
- Download only the necessary model
- First triage via the terminal
- Check versions and provenance
- Fix failures without rebuilding everything
Code, environment, and weights
What it is
The clone includes the SDK and the practical app; the weights are downloaded separately from Hugging Face. A virtual environment isolates the Python dependencies from the rest of the machine. The project keeps the upstream history and license, with the identified adaptation in the practical and docs files.
Why learn
Cloning the Git repo doesn’t automatically download the checkpoint. The first run can take much longer than inference with the resident model. Separate these two steps when describing the experience to someone else or when measuring latency.
git clone https://github.com/inematds/laya.git; then create the environment and install the package from the clone.
✓ Apply with criteria
It can download files, build the architecture, and load the weights. The lab’s inference measurement starts after this preparation.
✗ Avoid the automatic conclusion
Why does the first call take longer?
Don’t accept an answer just by the field name or by the appearance of precision. Check the definition and the context of this section.
Key concepts
versioned code
Python isolation
trained weights
downloaded files
Test your understanding
Why does the first call take longer?
Check the commented answer
It can download files, build the architecture, and load the weights. The lab’s inference measurement starts after this preparation.
Choose CPU or GPU deliberately
What it is
By default, the app uses CPU, so you can experiment without a compatible GPU. The --device cuda argument requests NVIDIA acceleration. The SDK can fall back to CPU in some failures; that’s why the result reports the device that was effectively used. Hardware type strongly affects latency.
Why learn
Don’t copy the published 33 ms to describe your machine. Record the Python version, PyTorch, transformers, device, number of questions, and input size. With a GPU, driver compatibility, architecture, and the PyTorch package matter as much as the amount of memory.
python3 -m practical --device cuda triage --message "I was billed twice."
-
1
Observe
python3 -m practical --device cuda triage --message "I was billed twice."
-
2
Define
The app uses CPU by default, letting you experiment without a compatible GPU.
-
3
Check
No. Check runtime.device in the response and the terminal warnings. The SDK has fallback paths to CPU.
Key concepts
general execution
NVIDIA acceleration
alternate device
real versions
Test your understanding
Does the cuda argument guarantee that inference happened on the GPU?
Check the commented answer
No. Check runtime.device in the response and the terminal warnings. The SDK has fallback paths to CPU.
Download only the necessary model
What it is
The adaptation preloads only multilingual, using the corresponding subdirectory from the model repository. This covers the lab in Portuguese without keeping three checkpoints in memory. The download command loads the model and confirms the device before opening the interface.
Why learn
Preloading all models may be suitable for varied traffic, but it increases resident memory. An app with a known language scope should start with the smallest necessary set. Don’t confuse parameter count with exact memory consumption during execution.
python3 -m practical --device cpu download
Reference command / schema
git clone https://github.com/inematds/laya.git
cd laya
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -e . -r practical/requirements.txt
python3 -m practical download
python3 -m practical triage --message "Quero meu reembolso."
Key concepts
preloading
model in memory
bundle variant
weights and intermediates
Test your understanding
What do you gain by running download before serve?
Check the commented answer
The files stay in the cache, and download problems show up before service. The process still needs to load the weights in its own memory.
First triage via the terminal
What it is
The triage command accepts a message and an optional subject, runs the four questions, and prints JSON. The result includes answers, routing, usage, runtime, and policy. Read the suggested department first, then the review signals and reasons; finally, check execution details.
Why learn
The command should exit with an error when the input is invalid or the model fails. This lets you integrate the tool into scripts without confusing unavailability with a valid recommendation. Preserve the JSON output of a successful test as evidence of installation.
python3 -m practical triage --message "I want to hire twenty licenses." --subject "New contract"
✓ Apply with criteria
None: the lab doesn’t refund money. automated_action_executed remains false and the operational action is human_review.
✗ Avoid the automatic conclusion
Which field indicates that money was refunded?
Don’t accept an answer just by the field name or by the appearance of precision. Check the definition and the context of this section.
Key concepts
command interface
process result
predictions
operational decision
Test your understanding
Which field indicates that money was refunded?
Check the commented answer
None: the lab doesn’t refund money. automated_action_executed remains false and the operational action is human_review.
Check versions and provenance
What it is
The fork starts from the upstream commit recorded in the documentation. The adaptation increments the version from 0.3.4 to 0.4.4 per the project’s convention, while keeping the patch as the minor is increased. Code, environment, and checkpoint are three different identities that must accompany an evaluation.
Why learn
If a result changes, compare those identities first. Remote weights can evolve even with the same repository name. For formal experiments, pin a model revision or keep a local snapshot and use --model-path to load it.
git rev-parse HEAD identifies code; the cache snapshot identifies the weights; pip freeze records libraries.
Key concepts
exact commit
weights version
library
repeat conditions
Test your understanding
Is it enough to record the laya-multilingual name to reproduce a result?
Check the commented answer
No. Also record the weights revision, code version, dependencies, schema, data, and execution conditions.
Fix failures without rebuilding everything
What it is
Start by identifying the failure layer: import, download, loading, tokenization, or forward pass. USE_TF=0 is an upstream recommendation to avoid TensorFlow probing in problematic environments. A network failure isn’t fixed by changing questions; an input error doesn’t require reinstalling the model.
Why learn
The lab exposes failures in the API and doesn’t invent a substitute classification. Keep the technical log in the terminal and an objective message in the interface. After fixing, rerun the smallest test that reaches the affected stage, followed by a real inference.
Download failed: test checkpoint access. Text exceeded tokens: reduce the input. Bad result: review the schema and evaluation.
Key concepts
find the layer
local intervention
short run
recognizable failure
Test your understanding
If the model classifies a text badly, should I reinstall everything?
Check the commented answer
No. First confirm the checkpoint, language, and scheme. Then compare with labeled examples; a reinstallation only makes sense if there’s evidence of a broken environment.
Module summary
- It can download files, build the architecture, and load the weights. The lab’s inference measurement starts after this preparation.
- The files stay in the cache, and download problems show up before service. The process still needs to load the weights in its own memory.
- No. Also record the weights revision, code version, dependencies, schema, data, and execution conditions.
Select a snippet from the lesson to highlight or annotate. Questions and notes stay in your journey; export the JSON to back up.