Francisco Lourenço

×

Roof Area Estimator from Aerial Imagery

This was an end-to-end project I developed for Roofr, focused on building a practical roof area estimation system from aerial imagery. The goal was to turn raw visual data into a reliable product workflow: starting from an address or geographic coordinates, the system retrieves the corresponding imagery, localizes the target property, understands the roof structure, and estimates the total roof area in a way that can be integrated directly into the client’s platform.

The final solution combines computer vision, geometry extraction, and measurement to operate under real-world conditions. It supports both Google Solar API and Google Maps imagery, allowing it to work across a much broader range of locations and data availability scenarios. When 3D or DSM information is available, the estimator uses it to reason more accurately about roof geometry; when it is not, the system falls back to a custom RGB-based estimation method designed to remain useful even without depth data.

My work covered the full technical backbone of the system, including geocoding and image retrieval, house localization, roof facet segmentation with custom neural networks, geometry refinement, area estimation, and the generation of client-friendly visual outputs such as overlays and rendered views.

A major part of the project was also building the data foundation from scratch. I collected and curated the imagery, designed the annotation guidelines and quality rules, and led an annotation team to create a high-quality training dataset for roof segmentation and estimation. I then used that data to retrain and improve the models, making the system much more robust to lower-quality imagery, denser urban scenes, and difficult real-world conditions.

To push the estimator further, I developed several advanced improvements, including alignment between Google Maps RGB imagery and Google Solar DSM data, a dual-imagery workflow, DSM-aided roof segmentation, and a redesigned Roof Estimator V2 with a new workflow and a manual mask drawing tool.

Overall, this project reflects the kind of work I enjoy most: building real-world measurement systems from visual data by combining machine learning, computer vision, geometry, dataset creation, and product-oriented engineering into a practical end-to-end solution.

📹 Roof area estimator platform in action

×

Aerial Roof Facet & Obstruction Instance Segmentation for 3D House Reconstruction

At Ellipse I worked on the computer vision backbone that turns aerial imagery into structured geometry for rooftop solar design. My focus was to reliably understand how a roof is shaped and what objects sit on top of it, so downstream tools can estimate usable solar area and generate accurate 3D models.

I developed two core segmentation systems:

• Roof facet segmentation – models that detect and separate individual roof planes (facets) from high-resolution imagery and elevation data, providing clean polygons that can be directly consumed for 3D reconstruction and solar layout.

• Object & obstruction segmentation – dedicated models for rooftops elements (chimneys, skylights, vents, HVAC units, nearby trees, etc.) to automatically flag areas that reduce usable panel surface or cast shadows.

To support these models long term, I also designed and built a new in-house dataset for Ellipse:

I created the annotation guidelines, class taxonomy and quality checks, then coordinated and reviewed labeling so that every polygon, facet and obstruction was consistent with how the models and solar design tools would consume them. The result is a production-ready dataset and segmentation stack that Ellipse can continue to expand as they scale to new cities and imagery sources.

🧠 Roof facet and obstruction segmentation results. Each image has original on the left, ground truth in the center and inference on the right.

×

City-Scale Point Cloud Colorization & Georeferencing Pipeline

EyeVi Technologies is a geospatial, AI-powered startup offering an on-demand mobile mapping solution for road network intelligence. By combining 360° imagery, LiDAR, and GNSS/INS data, EyeVi helps cities and infrastructure owners inspect roads more efficiently and build high-detail digital twins.

I joined the team as a 3D ML/CV Engineer, where I worked on the all-in-one processing pipeline that transforms raw sensor logs from EyeVi’s survey vehicles into coherent, city-scale street-level datasets. Each drive generates large amounts of heterogeneous real-world data, including 360° RGB panoramas, dense LiDAR scans, and vehicle trajectories produced by the navigation stack.

My work focused on designing and implementing a substantial portion of this production pipeline. I was responsible for roughly half of the full production system, including the final client-facing stages that transformed processed sensor data into deliverables used by EyeVi’s customers.

My contributions included building systems to:

• Ingest and validate raw survey logs
• Colorize LiDAR point clouds using 360° imagery
• Georeference outputs into a consistent global coordinate system
• Generate orthostrips: high-resolution top-down road mosaics created by stitching ortho photos
• Develop visualization, calibration, and quality control tools for the processing team

A key part of the pipeline involved synchronizing LiDAR data, panoramic imagery, and trajectory information so the different sensor streams could be processed consistently and used reliably in downstream mapping workflows.

One of my main contributions was the development of the colorization and sensor fusion pipeline, where LiDAR points are transformed into the corresponding camera frames and projected onto panoramic imagery to recover RGB values. This required handling overlapping observations, visibility constraints, out-of-field points, and other real-world challenges to produce high-quality colorized point clouds that preserve both geometric accuracy and visual context.

Because the colorization pipeline was the final stage of the overall system, the outputs of my work were not just internal processing artifacts, but part of the actual products delivered to clients. This made the work especially important from both a technical and product perspective, since it directly affected the quality, usability, and visual value of the final deliverables.

I also worked on the georeferencing layer, propagating vehicle pose information through the processed data so that outputs could be anchored in a consistent global coordinate system. This is essential for EyeVi’s core applications, including road asset inventories, condition monitoring, and digital twin generation, where spatial consistency across surveys and external GIS data is critical.

In addition, I developed visualization, calibration, and quality control tools used by the processing team to inspect data quality, validate sensor alignment, and support the reliable operation of the wider mapping pipeline. These tools helped make the workflow more practical and robust for production use, while improving visibility into issues that could affect downstream outputs.

I also extended the pipeline to generate road orthostrips as a standard product. Using georeferenced imagery and trajectory-aware processing, the system produces high-resolution ortho photos of the road surface and stitches them into long, continuous strips aligned with the vehicle path.

This work sits at the core of EyeVi’s mapping stack and supports many of the downstream AI and analytics tools delivered to clients. More broadly, it reflects the kind of work I enjoy most: building end-to-end computer vision and 3D data systems that turn complex real-world sensor data into practical, reliable outputs.

📹 3D Reconstruction of a street using the all-in-one pipeline and visualized on the QC tool developed by me using potree

×

3D Shape Classification System

This was a fully end-to-end solution I designed and delivered for Anything World. The objective was to develop a robust classification system to support both the Anything World API and the Animate Anything tool, which enables automatic rigging and animation of 3D characters.

Since different character types require distinct animation behaviors (e.g., a human moves on two legs, a dog on four, and a car uses spinning wheels), understanding the character's category is essential prior to animation. The tool supports over 50 categories, making accurate classification a foundational requirement.

I was responsible for researching, developing, and deploying a classification pipeline capable of taking an unaligned 3D mesh as input and outputting a single predicted class from the supported set.

After an in-depth literature review, I implemented and adapted a state-of-the-art multi-view graph convolutional neural network approach. The method renders multiple images of the 3D shape from evenly spaced viewpoints, extracts global features per image, and connects them in a graph structure processed by a GCN.

I introduced several custom modifications to the architecture to optimize performance and make it rotation-invariant, addressing the core challenge of working with unaligned meshes. These adaptations were specifically designed to meet the needs of real-world deployment within Anything World’s production environment.

The final model achieved a 95.6% F1-score on the rotated ModelNet40 benchmark — which, to the best of my knowledge, was a state-of-the-art result on this dataset at the time (2023).

📹 Classification system demo on the Animate Anything tool

×

3D Shape Automatic Alignment

This was a fully end-to-end project I led at Anything World, aimed at developing an automatic 3D alignment system to support both the Anything World API and the Animate Anything tool.

The objective was to rotate unaligned 3D characters into a canonical orientation before they were passed into the rigging and animation pipelines. Since these systems are rotation-variant, their performance was highly dependent on the initial pose of the character — making consistent alignment a critical pre-processing step.

I was tasked with researching, designing, and deploying a system that would take any unaligned 3D mesh as input and return a rotation matrix aligning the model to a predefined canonical pose.

After a comprehensive literature review, I found that existing methods did not meet the accuracy demands of our production requirements. As a result, I developed a custom hybrid method combining ideas from several academic approaches with in-house techniques I designed.

The resulting system — a proprietary method called “Symmetry-Aware and Base-Plane-Aided 3-Step Rotation Estimation” — achieved outstanding results. On a dataset of 220 diverse 3D characters across 56 categories (including people, animals, vehicles, and objects), it achieved a median mean angular error (MAE) of just 2.47°, meaning the system delivered near-perfect alignment in the vast majority of cases.

This solution is now a key component of the Anything World pipeline and is used in real-time for character animation and rigging in production.

📹 Automatic alignment system demo on the Animate Anything tool

×

3D Shape Part Segmentation System

This was another end-to-end project I developed at Anything World, focused on 3D shape part segmentation — a key step in the automatic rigging and animation of 3D characters. The objective was to identify the precise location of specific semantic parts (e.g., arms, legs, body) within 3D models to enable accurate joint placement and deformation.

Unlike the classification and alignment tasks I previously solved, a segmentation system was already integrated into both the Animate Anything tool and the Anything World API. My role was to either optimize the existing solution or design a more effective one from scratch.

After several iterations attempting to improve the current model, we determined that it had already been optimized to its full potential in terms of accuracy. As a result, I conducted a new round of literature review, identified a state-of-the-art method with potential advantages, and implemented a fresh solution for evaluation.

The newly developed model achieved similar accuracy to the existing system, thus revealing a likely dataset bottleneck. However, the key breakthrough was that my solution was significantly more efficient — offering reduced training time and substantially faster inference, which aligned better with our production requirements.

The final system was based on a graph convolutional network (GCN) for point clouds, enriched with ideas from multiple research papers. It achieved 88.8% mIoU on the ShapeNet Part benchmark — a competitive score, on par with the top-performing models on this dataset.

Thanks to its performance and speed, this new solution was successfully deployed into Anything World's production pipeline.

🧠 Part segmentation results across different categories

×

Custom Multi RGB-D Camera System Based 3D scanner

coatingAI is a startup pioneering the use of AI in the powder coating industry, with the goal of improving efficiency and reducing material waste while maximizing quality and precision.

A critical component of their pipeline involved obtaining a high-fidelity 3D understanding of the parts entering the powder coating line. I was brought on as their first 3D Computer Vision Engineer and was tasked with researching, designing, and deploying a custom multi-camera 3D scanning system — built entirely from scratch.

The system was based on multiple RGB-D cameras capturing metallic parts in motion, which were later fused into a full 3D model. Since I initially worked fully remotely from Coimbra, Portugal, I built a physical replica of the powder coating line in my office to prototype the solution. This setup included a metallic structure with movable mounts, allowing me to simulate the part transport mechanism and validate the camera placements.

In addition to the hardware design, I developed the full software pipeline responsible for processing the RGB-D data, estimating camera positions, and generating an accurate 3D scan of each part as it moved through the system.

The final solution delivered precise and consistent 3D reconstructions of metallic parts in real-time, and was successfully deployed into coatingAI’s production environment — enabling downstream automation for coating quality optimization.

📹 My first months working with coatingAI: Here you can see the first 3D scanner prototype and the sketchy metal structure I built to replicate the coating line.

×

Anomaly Detection System for a Glass Bottle Production Line

This was a research-focused project I contributed to at the Institute of Systems and Robotics (ISR), University of Coimbra, aimed at developing a computer vision-based anomaly detection system for a glass bottle production line.

My role centered on two key areas:

• Designing and implementing the anomaly detection software responsible for identifying visual defects or irregularities in real time.
• Contributing to the optimization of the camera setup, including the number, placement, and calibration of cameras to maximize inspection coverage and accuracy.

This project combined elements of classical and learning-based vision approaches, and was designed to operate under the high-throughput and precision requirements of industrial production lines.

×

6DoF Object Pose Estimation from RGB-D Images Using Machine Learning Approaches

×

Intel RealSense SR305, D415 and L515: Experimental Evaluation and Comparison of Depth Estimation

×

Evaluation of the Accuracy of Pose Estimation Based on Relative Pose