Dermi

Objective: 

To develop a freely accessible mobile application leveraging artificial intelligence (AI, specifically a CNN pipeline) to detect potential signs of skin cancer in uploaded images of skin lesions, allowing users to seek timely medical attention and raising awareness about early detection.

Project Scope:

This project will involve the development of a mobile application with the following features:

• Image capture and upload functionality for skin lesion analysis.

• CNN-powered (AI vision) assessment of uploaded images for potential skin cancer indicators.

• Educational resources about skin cancer prevention and early detection.

• Secure handling of user data and privacy compliance 🕵️

Methodology:

1. Data Collection:

• I'll use publicly available skin cancer image datasets (e.g., ISIC Archive, MNISC HAM10000, DermNet).

• Focus on training the AI model to differentiate between benign and malignant lesions (maybe later I might create a final model version to differentiate between specific skin diseases/conditions, but this will require a lot of high-quality data in many specific categories)
  
  

2. Model Development:

• I'll use a CNN architecture w/ transfer learning on pre-trained models (likely a model from TensorFlow Keras). 

• Fine-tune the model using the curated dataset (train, test, val classes) for accuracy in detecting skin abnormalities. Aim for 90%+ accuracy, if not higher (on final testing and validation). 
  
  

3. App Development:

• Develop a user-friendly mobile application interface using React Native.

• Implement educational sections providing tips on skin health and prevention.

• Integrate the model prototype into the app and develop back-end processing (see below) + ensure user image privacy while still balancing app personalization. 
  
  

4. Backend and Deployment:

• Host the trained AI model on a cloud platform (e.g., Google Cloud or AWS?). 

• Build a backend API to process images and return analysis results to the app. 
  
  

5. Testing and Validation:

• Evaluate model performance (in Kaggle + on app) using metrics such as accuracy, sensitivity, specificity, and F1 score. 

• Perform clinical validation by comparing model predictions against dermatologist diagnoses (work w/ Dr. Brogan or another clinic on this?). 

Full Technology Stack 

Here's the technology stack I used for this project: 

• Frontend: WebStorm IDE, React Native, Figma (for design)

• Backend: JavaScript + Python

• Machine Learning Algorithm: TensorFlow+ Kaggle (Python), Render (API deployment), GitHub for organization.

• Cloud Hosting:  Amazon AWS 

My first step was to learn about the classifications of skin cancer, existing detection methods, and current AI diagnostic tools. After a long, arduous but highly riveting review of over 200 (247, to be exact) different papers and tech reports, I wrote a *brief* summary of my findings: the whole doc includes all the paper summaries, and I've linked it to a PDF below. Do brace yourselves, as it is tremendously long and has an absurdly, ridiculously, exceedingly large amount of words. 

All of this information was important for my algorithm and overall tech development (including later project extensions), since the goal of my project was, after all, to improve existing technologies/conditions surrounding image-based skin cancer detection. 

All in all, the entire literature review and research process took around 2-3 months (similar to my studies on coding and artificial intelligence as I do mention later, some of this occurred concurrently with project development). 

Initial App Design*

I began by analyzing existing AI-driven image recognition apps designed to identify dermatological conditions. In total, I reviewed nine apps—eight available on the Google Play Store and one, SkinVision, developed by a Swiss company and not accessible in the U.S. or Canada. Among these nine apps, seven featured paywalls, with costs ranging from $5 per scan to $29.99 per year. Quite horrendous, I am aware. The average combined rating across all apps was 3.91 stars, with four apps rated below 4 stars and one lacking a rating altogether. Additionally, many of these apps exhibited issues such as absurdly odd grammatical errors and challenges related to UI design, including cluttered layouts and difficulties with uploading scans/reviewing results. 

I planned to first build a functional rudimentary version of DermaScan, and then work towards a final version later. 

Here's the simple user flow for the rudimentary version of DermaScan:

1. User opens the app. 

2. Lands on Home Screen with 3D model display (non-interactive) ⇒ clicks “Upload Image/Take a Photo” or “History”

3. Navigates to Image Upload screen ⇒ uploads an image (either from camera or device files). Receives a risk assessment and possible diagnoses. 

4. Navigates to “History” ⇒ displays record of past scans (both front and back). 

5. Optionally explores Settings + customizes profile. 

I then created mock-ups of specific app pages for the final version of DermaScan using Figma (a web-based user interface design platform). Below this section are samples of my work. 

*The initial name "DermaScan" was changed to "Dermi" after this stage. I thought "Dermi" was, to be frank, cuter. 

React Native + Learning by Doing

To build the rudimentary Dermi prototype, I decided to use the Javascript framework React Native. I chose this framework because it allows for cross-platform development (i.e. allows the app to run on both iOS and Android using a single codebase). Having never worked with React Native prior to the app build, I first spent a month or so learning the framework through various Youtube courses. In the process, (and because I'm an ardent proponent of learning-by-doing), I practiced my skills by creating a functional Movie Finder application through this tutorial. 

After I felt sufficiently confident in my skills, I spent around 2 weeks building the first version of a rudimentary, user-interactive prototype for Dermi. Initially, the functionality was limited only to 3 main pages and was deliberately minimal: Help, Home, and History, as well as both bottom tab navigation and menu navigation via a pop-out sidebar. 

3D Body Model: Using Blender

Coming into this project, I had envisioned an interactive 3D model of a human body that a user could select; e.g. long pressing the right arm of the model to prompt a scan on that exact body part. To do this, I decided to use the Blender software. Since I had never used Blender before, cue another round of YouTube tutorials. The learning curve was steep but essentially just a buffed-up version of Google Drawings (which I’d used frequently in 5th grade as the head of our school newspaper). Learning how to use Blender was frustrating, but eventually allowed me to pick up Fusion360 relatively quickly as well (see project extension - DermiScope). 

The hardest part about constructing the 3D model was definitely the facial proportions, and I had to start from scratch at least 2 times. All in all, the (male) 3D model took around 1 week or so to build and ended up being my pride, joy, and one of my favorite parts of the entire process. Later, I downloaded an open-source female model to give users gender options as well, since not everyone identifies as a gigantic, muscular blue man. 

Implementing the 3D model into my app proved to be equally, if not more, difficult. I’ll talk about this process a bit more in the final version, but the actual implementation occurred during this first app prototype phase. 

Coming into this project, I knew no more about artificial intelligence and machine vision than did the common, ignorant man. To me, AI was synonymous with the robots from Star Wars and the imminent global synthetic humanoid takeover. I, like a true Neanderthal, had absolutely no idea how something made out of non-life could even come close to exhibiting human-like or intelligent behaviors. In fact, understanding such information seemed daunting and borderline impossible. Needless to say, this was far (okay, only a measurable distance) from the truth. 

To rectify my ignorance and further develop the necessary knowledge to conduct my project, I sat my buttocks down at the desk for around 4-5 months (it should be noted that after I had learned an adequate amount of material to support my project goals, much of my further study occurred concurrently with actual project development) and intensely immersed myself into the field of machine learning, deep learning, and various other aspects of AI: personally, I tend to learn best when I'm forcibly thrown into the deep end. What ensued, you might wonder, was nothing short of a man-made miracle: to be speak genuinely- I was enlightened. 

As my infantile mind delved into the fascinating world that is AI (namely, through various open source courses including MIT 6.S191 - Deep Learning and Practical Deep Learning for Coders by Fast.Ai, as well as books/textbooks including Deep Learning With Python by François Chollet, Why Machines Learn: The Elegant Math Behind Modern AI by Anil Ananthaswamy, and Advanced Deep Learning with TensorFlow 2 and Keras by Rowel Atienza), I began to slowly unravel, string by string, the beguiling complexity behind what I had previously dismissed as mere science fiction. 

The first major revelation I underwent came upon my understanding that AI is not magic: in fact, it is mathematics in its purest form. Specifically, artificial intelligence is but an organized jumble of linear algebra, calculus, and statistics working in joint to find patterns in data that would be near-darn-impossible for humans to detect manually. When I first grasped how a convolutional neural network could automatically learn to identify edges, textures, and increasingly complex features in images through backpropagation, I legitimately felt as if my Neanderthal mind had discovered fire. It was seductively fascinating, the concept that a machine could essentially teach itself to see, not through pre-programmed rules, but through exposure to hundreds or thousands of examples. This was to me, might I say, of greater revolutionary proportion than the American, French, Haitian Revolution, and all threats to 1700s monarchy combined. 

Even after learning all the relevant parts necessary to my project (CNN's, GAN's, transfer learning architectures, how to build models using software, etc), my thirst for intellectual evolution was unquenched and my fascination for the subject yet undying. Thus, I decided to continue my study (while also beginning to build my model for Dermi, the process of which I've outlined in the succeeding section) and delved into other topics within the field of deep learning, including advanced optimization algorithms, reinforcement learning paradigms, natural language processing architectures, the mathematical foundations underlying transformer models, etc. My exploration extended to current research papers, where I read more on attention mechanisms, regularization techniques, the theoretical underpinnings of gradient descent optimization, and other various delightful tidbits. 

Over the course of my studies, the Neanderthal was transformed into a modern-day human: I ended up filling over 3 complete laboratory notebooks with notes, diagrams, mathematical derivations, and code snippets, totaling approximately 400 pages of handwritten documentation. At the end of my evolutionary pathway, I could confidently proclaim that I'd acquired far more than the technical knowledge necessary to build a CNN-based skin cancer detection algorithm for Dermi. Beyond that, and probably more crucially in personal regard, I also felt that my brain had been fundamentally rewired to understand what exactly is possible at the astonishingly spectacular frontier that is our current tech industry. 

Overall, my assiduous journey from complete ignorance regarding AI to building actual neural networks and understanding deeper technical concepts further proved to me that there are no intellectual barriers that cannot be demolished with enough curiosity, persistence, and regular studying (to this day, my hand cramps at the thought of those 3 notebooks). 

I conclude this project phase with this thought: if a Neanderthal 16-year-old like I may go from thinking AI as comparable to Hollywood nonsense to actually architecting models, then truthfully, whatever is stopping anyone from diving headfirst into whatever field intimidates them most? With the sincerest of all honesty, the beauty of our modern educational landscape is that MIT-level courses, research papers, and gold-standard textbooks are only but a Google search away. This is, in my humble opinion, something we ought to be both immensely grateful for and seeking to take the full and utmost advantage of, by anyone and everyone, to our curious hearts' desire. 

1. Data Sourcing and Curation

• Data Sources: The model was trained on a consolidated dataset I named dermi_final, which was an amalgamation of several public, gold-standard dermatological archives:

  • • • • The ISIC Archive (2019 & 2020): The International Skin Imaging Collaboration archive, a large-scale repository of dermoscopic images used in academic research worldwide.

        • HAM10000: A extensively curated dataset of 10,015 dermatoscopic images covering a wide spectrum of common pigmented skin lesions.

        • DermNet: One of the largest online dermatology atlases, providing a diverse set of clinical (non-dermoscopic) images, crucial for ensuring the model works on typical smartphone photos.

• Combining these datasets was a strategic decision to introduce significant variance in:

  • • • • Imaging technique: both dermoscopic (magnified, uniform lighting) and clinical (standard camera, variable conditions) images.

        • Demographic representation: images from a wide range of skin tones and anatomic locations.

        • Lesion morphology: a comprehensive representation of both common benign lesions (e.g., nevi, seborrheic keratosis) and malignant ones (e.g., melanoma, basal cell carcinoma).

• Data Integrity: Duplicate images across datasets were programmatically identified and removed to prevent data leakage and overfitting, resulting in a final, pristine dataset of 77,313 images.

• Data Split: The dataset was partitioned to rigorously evaluate model performance:

  • • • • Training Set: 70% (~54,103 images) - used to train the model.

        • Validation Set: 15% (~11,597 images) - used to tune hyperparameters + evaluate during training.

        • Test Set: 15% (~11,613 images) - used only once for the final, unbiased evaluation of the model's real-world performance.

2. Data Preprocessing and Augmentation

• To maximize the model's ability to generalize from the training data, a robust preprocessing pipeline was implemented.

• Rescaling: All pixel values were normalized from a range of [0, 255] to [0, 1] to stabilize and accelerate the training process.

• Resizing: All images were resized to 224x224 pixels, the standard input size for the pre-trained architectures used.

• Data Augmentation: A critical step to artificially expand the dataset and teach the model to be invariant to irrelevant variations. The training data was augmented in real-time with aggressive transformations to simulate various photo-taking conditions:

  • • • • Geometric Transformations: Rotations (±45°), width/height shifts (±30%), shearing, and zooming.

        • Photometric Transformations: Adjustments to brightness (±30%) and random hue variations to account for different lighting and skin tones.

        • Flipping: Both horizontal and vertical.

3. Algorithm Selection and Architecture Evolution

• Why CNNs?

  • • • • Convolutional Neural Networks (aka CNNs) are the state-of-the-art for image analysis in this time and age. Their architecture, with convolutional and pooling layers, is inherently designed to hierarchically learn spatial features (early layers = simple edges and textures, deeper layers = more complex patterns and shapes), thus making them perfectly suited for identifying patterns in skin lesions.

• Why Transfer Learning? 

  • • • • Training a massive CNN from scratch requires immense computational power and data, both of which I do not have. Transfer learning allows us to leverage a pre-trained model (like ResNet50V2) that has already learned rich, generic feature representations from millions of images (ImageNet). Then, we can adapt this powerful feature extractor to a specific task, namely skin lesion analysis, with relatively less data and power.

Dermi Architecture Iteration:

1. First Version: EfficientNetB4

I started with a modern, highly efficient architecture known for its excellent accuracy-to-parameter ratio. The goal was to establish a strong baseline performance.

• Process: The base model was loaded pre-trained on ImageNet. Only the last 10 layers were unfrozen for fine-tuning, with a simple classifier head of two dense layers added on top.

• Result: This model performed remarkably well, achieving ~97% validation accuracy and a ROC AUC of 0.991. It proved the concept was viable. However, high accuracy can be misleading in medical imbalanced datasets; a deeper look was needed. 

2. Second Version: ResNet50V2

I switched to the ResNet50V2 architecture for its proven robustness and residual connections that help with training deeper networks: the focus here shifted from pure accuracy to clinical utility.

• Key Improvements:

• Custom Loss Function: I implemented a Clinical Focal Loss to focus learning on hard-to-classify examples.

• Clinical Metric: I programmed a custom malignant_recall metric to explicitly monitor the model's sensitivity during training.

• Class Weighting: Applied a 25% higher weight to the malignant class.

• Result: This model was a valuable experiment in clinical prioritization. However, its validation metrics were unstable between epochs, and the custom loss function introduced complexity that made training less predictable. It achieved a high malignant recall but at a significant cost to overall precision and accuracy. Quite unfortunate, but not exactly a failure. 

3. Final Version: InceptionV3

For the final model, I selected the InceptionV3 architecture. Its design, using inception modules that process data at multiple scales simultaneously, is exceptionally well-suited for analyzing lesions where features of varying sizes (fine textures, larger structures) are all important. It also has an nicely ideal balance between the efficiency of EfficientNet and the robustness of ResNet, typically leading to superior + more stable performance on datasets.

• Key Improvements:

  • • • Input Size: I changed the input dimensions to 299x299 pixels to match InceptionV3's native optimal size, allowing the model to leverage more granular image details.

      • Structured Fine-Tuning: I unfroze only the last 30 layers of the base model for training. This provided a more balanced approach: the early layers retained their general feature-detection capabilities, while the deeper, more specialized layers were adapted specifically for skin lesions.

      • Robust Classifier Head: I also designed a deeper, more powerful head with three dense layers (1024 -> 512 -> 256 units), each followed by Batch Normalization and Dropout (0.3). This architecture is far more capable of learning the complex decision boundary between classes without overfitting.

      • L2 Regularization: I added L2 regularization to every dense layer to further penalize overly complex weights and improve generalization.

      • Simplified, Effective Loss: I decided to return to a standard Binary Cross-Entropy loss, which proved more stable and reliable than the custom focal loss in this architecture. Combined with automatically computed class weights ({0: 0.59, 1: 3.25}), it effectively handled the class imbalance by giving more weight to the minority malignant class.

      • Performance Optimizations: I further icorporated steps_per_execution=5 and prefetch(tf.data.AUTOTUNE) to maximize GPU utilization and drastically reduce training time per epoch. 

• Result: The final InceptionV3 model demonstrated superior training stability and strong, balanced performance. It converged smoothly, as shown in the clean accuracy and loss curves. Most importantly, it achieved an excellent balance on the test set: 83% accuracy, a 87% recall (sensitivity) for malignant lesions, and a very high 97% precision for benign lesions. This means it is highly sensitive in catching potential cancers while also being very reliable when it assures a user a lesion is likely benign. While the end product is far from perfect and definitely not 100% clinically up to par, I decided to move forward with the next phase (i.e. make it exist first, and then make it perfect). 

After the development of a relatively sound benign/malignant differentiating model, I decided to move forward with implementing it into a second Dermi app prototype. To clarify, I was fully aware that the initial model version I implemented into the app prototype at this stage would not be the final, published model version. However, I decided it practical to return to further model refinement (for final launch) only after building a final app prototype with full functionality. This was decision was primarily because putting things off gives me anxiety, and I also could not stand looking at that ugly, non-functional initial app prototype for a moment longer.  

So first, I downloaded the "final" version of the model in the last stage as a .keras file (373 MB). I then uploaded my model to AWS S3 as a Bucket, which I then linked to Render, a cloud application platform I used to set up the API. As simple as all that sounds, this quite literally took me 5 days to do 😐

But as always, things didn't go quite according to plan. The initial deployment attempt on Render failed immediately due to a memory constraint. The logs clearly indicated the issue: ==> Out of memory (used over 512Mi). The free tier on Render provides a maximum of 512MB of RAM, and the standard TensorFlow library, along with my 373MB model being loaded into memory, easily exceeded this limit during execution.

My first instinct was to try and reduce the model's size because truthfully, I would much rather eat a gallon-sized can of toxic, expired raw beets immersed in a rancid Carolina Reaper hot sauce than have to pay for an API. 

So first, I replaced the full tensorflow package in my requirements.txt with the lighter tensorflow-cpu variant. This successfully reduced the build size, as evidenced by the logs showing a download of 230.2 MB for tensorflow-cpu instead of the previous 615.4 MB for the full package. 

However, this was only a partial victory. While the build stage passed, the deployment stage failed yet again with the same Out of memory error. The application was still loading the 373MB .keras model into RAM on startup, which alone consumed nearly the entire 512MB allocation before processing a single request. Ergo, it became clear that the model itself was the primary issue here. 

Plagued with ominous hallucinations of toxic beets and rancid hot sauce, I decided to convert my raw model into a TFLite version. If you're unfamiliar with what TFLite means, it is essentially just a slightly-compressed format of an AI model designed to run efficiently on mobile, embedded, or IoT devices.

My plan was straightforward:

• Convert the large .keras model from AWS S3 into a quantized .tflite model locally.

• Upload this new, smaller .tflite model back to S3.

• Modify the Render API to use the TFLite interpreter instead of the standard Keras model loading.

• Code the React Native app to handle TFLite model outputs.

However, the conversion process, which I assumed would be a simple script, unexpectedly morphed into a multi-day compatibility war in which my brain was nearly drop-kicked into an alternate dimension of insanity. 

• My initial attempts to run a conversion script failed with a ImportError: DLL load failed, indicating a broken TensorFlow installation. After some investigation, I determined the root cause to be my primary Python installation, version 3.13, which was too new and not yet supported by TensorFlow. To fix this: I installed Python 3.11, created a fresh virtual environment explicitly using the Python 3.11 interpreter, and, within this new environment, installed a compatible TensorFlow version (2.14.0).

• A new error then emerged: AttributeError: _ARRAY_API not found. This was due to a newly released NumPy version (2.2.5) that was incompatible with TensorFlow 2.14. The solution was to downgrade NumPy to a compatible version (<2.0). 

• Finally, with a stable environment, the script ran but hit a deserialization error: ModuleNotFoundError: No module named 'keras.src.models.functional'. This was caused by a Keras installation conflicting with TensorFlow's built-in tf.keras module, which I fixed by simply uninstalling. 

After overcoming these disgustingly frustrating environmental obstacles, the conversion script executed perfectly! The results were quite pleasing, with a reduction from 373.3 MB (original model size) to a TFLite model size (FP16 Quantized) of 93.5 MB: a 75% reduction. 

With the newly converted dermi_model.tflite file uploaded to AWS S3, I made some changes to the Render API (such as using lazy loading), replaced the standard Keras model loading logic with the TFLite Interpreter, and pushed the changes to GitHub. I then, after many YouTube tutorials, wrote a client-side script to successfully implement the model API. 

Aftering successfully incorporating the API into my first app prototype, I decided to move forward with building the full app. Below are details on each of the major screens/functionalities within my final app version:

Home Screen:

I decided to have the home screen as the core screen of the app: the home screen is where everything converges, and displays recent scans (which I fetched from the History screen) with their status indicators (benign in green, concerning results in orange), quick actions for immediate navigation, and real-time stats showing total scans and analysis progress. 

The tricky part was managing the pending scans system. When users start a scan analysis, it can take a while for the API to respond, so I decided to build a background processing system that tracks analysis status and updates the UI accordingly. Thus, the home screen intelligently handles these pending states and also shows processing indicators, allowing users to continue in the background without losing their place.

NewScan Screen:

This is the screen I had initially envisioned: the NewScan screen features the interactive 3D body model I built, complete with zoom controls, rotation, and tap-to-select functionality. On the screen, users can either:

1. Select a specific body part from the 3D model (by long-pressing), which opens the BodyPartModal.

2. Use the general "Take Photo" button for unknown area scans via CameraModal. 

Both modals handle the complete image capture and analysis workflow in the exact same way: users can either opt to take a picture using their local camera (after accepting camera usage permissions from Dermi), choose an image from their local file system (also after accepting usage permissions from Dermi), or cancelling the modal. 

I also implemented sophisticated error handling here: if a scan analysis takes too long, users can continue in background mode by X’ing out the modal and get notified when results are ready (wherein the ScanResultScreen automatically opens, overlaid on the History screen). More on this later. 

The 3D model responds to touch gestures with momentum-based rotation, pinch-to-zoom, and automatic highlighting of body parts on hover (this was a huge pain in the buttocks to code). It even has an auto-rotation feature that kicks in when users aren't interacting with it, which I implemented just because I thought it would look cool (spoiler: it looks really cool). This step, although seemingly simple, probably took me ~2 weeks of intense debugging and trial by error, especially since I couldn’t find any exact tutorials on how to integrate an interactive 3D model into a React Native app. 

History Screen:

The History screen handles both completed scans and pending analyses. It was crucial to get the state management right here since scans can be in various states (analyzing, completed, failed), and the UI thus needs to reflect this accurately. 

To meet this requirement, I implemented a smart refresh system that checks for completed background analyses and automatically navigates users to results when ready (i.e. displaying the ScanResultScreen automatically, which then, if closed, displays the History screen). The history items show translated body parts, formatted dates, and status indicators, all while maintaining smooth performance even with large scan histories :)

ScanResult Screen:

While not an actual screen on the app, the ScanResultScreen represents the finalized report from the entire scanning process and shows the users their analysis results through a visually-pleasing, medical-report-style presentation.The screen features a cute little animated circular progress indicator that draws around the scan image, creating a satisfying visual completion effect.

Key features of the report include:

• Animated results presentation, with the scan image appearing within an animated circular border that fills in over 1.2 seconds, followed by a subtle pulse animation to indicate completion. 

• Border colors change based on results: green for benign findings, orange for suspicious. 

• Display of body part location of scan, condition assessment, confidence percentage, detailed descriptions, and further recommendations. 

• Multi-language support: all content adapts to the user's selected language, including medical terminology and recommendations (see more in Technical Architecture - Internationalization). 

Downloadable Report Functionality:

The most technically challenging aspect here was implementing the downloadable report feature. This required creating a complete PDF-like report that works across both web and mobile platforms: 

• For web implementation, I used html2canvas to generate a complete HTML report with professional styling, which is then converted to a downloadable PNG. 

• For mobile implementation, I used react-native-view-shot to capture a hidden, full-width report layout optimized for mobile viewing.

The generated report also includes a professional medical document layout with:

• Header section with report title and generation date.

• Patient scan image with appropriate border styling.

• Complete analysis results in a neat grid format.

• Detailed descriptions and recommendations sections.

• Medical disclaimer footer emphasizing the need for professional consultation.

The technical implementation included the maintenance of a hidden ViewShot component that renders a full 800px-wide report layout specifically optimized for screen capture (this step took a lot of trial and error, and you can see some of my downloaded report “failures” below this phase section). This separate rendering ensures the downloaded report has proper formatting, regardless of screen size. For web users, it generates HTML with embedded CSS styling, handles image loading asynchronously, and creates downloadable links.  

Integration with History Screen System:

The ScanResult screen further integrates with the History screen through: 

• Automatic Navigation Flow: when background scan processing completes, the system automatically opens the corresponding ScanResult screen overlaid on the History screen. Users can view their results immediately, and when they close the results, they're returned to the updated History view. 

• State Persistence: the screen intelligently determines whether it's showing a new scan result or a historical one, adjusting the saving behavior accordingly. New scans are automatically added to history, while historical scans are simply displayed without duplication (this was primarily for the testing phase). 

• Smart Date Handling: for older scans, the system retrieves the original scan date from storage (again, this was only for the testing phase- all new scans follow the succeeding process); for new scans, it uses the current timestamp, ensuring accurate chronological organization in the History screen. 

Help Screen:

The Help screen has multiple support options, including: 

• Interactive scan guide with step-by-step instructions and visual examples (credits to Sora + OpenAI for generating these images). 

• Comprehensive FAQ system covering common questions.

• Direct access to DermiBot for real-time assistance.

• Email support integration.

The scan guide is particularly polished and is basically a modal-based tutorial with progress indicators, navigation controls, and tips for getting better scan results (i.e. camera positioning, lighting, etc). 

Explore Screen: 

This screen was one of my favorites to build. I had decided that the initial 3 bottom nav screens (Home, New Scan, and Help) were insufficient, so I decided to beef up my app by adding in even more features. One of these features includes the Explore screen, which is essentially a comprehensive dermatology database with over 60 different skin conditions, from cancerous to benign. Each condition includes highly accurate information and details obtained from DermNetNZ.org aka DermNet, an accredited global clinical resource website for dermatology and various skin conditions. Each condition on my database includes: 

• Localized names and descriptions.

• Associated symptoms and body parts.

• Treatment information and risk factors.

• High-quality medical images.

Users can search by condition name or filter by symptoms using an interactive symptom bubble interface. The search is intelligent: it handles typos, partial matches, and works across multiple languages as well. 

I also integrated a blog system (an AllArticlesScreen and ArticleDetailScreen) here with educational articles about skin health, sun protection, and early detection. Each article supports multiple languages and includes content with proper typography. 

Find Screen:

The Find screen was also fun to build. The screen has the functionality to locate nearby dermatology clinics using a custom backend API (which I also set up with Render). I also got creative with the UX: the screen includes predictive location search with real-time suggestions as users type city names or zip codes, a bit similar to how Google Maps Search works. The location suggestions use OpenStreetMap's Nominatim API with intelligent filtering for US locations. As users type, they can see real-time feedback with properly formatted suggestions showing city, state, and zip codes. The system also handles both city names and postal codes.

Clinic results show distance (converted to miles, or km depending on language selection in Settings- see below), clinic ratings (out of 5 stars), current hours or availability, and direct integration with native map apps for directions. 

Settings & Profile: 

The Settings screen manages app preferences, language selection, and comprehensive privacy/terms documentation for Dermi. The language selector supports 8 languages with proper right-to-left text support for Arabic (more details on this in Technical Architecture below).

The Profile screen includes demographic information (toggle-able by the user, which includes an age selection modal, gender selection, and race selection) and an embedded Fitzpatrick skin type test below. This (commonly used and scientifically sound) test determines users’ skin sensitivity to UV exposure through a series of questions about natural skin color, hair color, eye color, and sun reaction patterns. 

DermiBot: AI-Powered Chatbot Assistant

DermiBot represents one of the more technically sophisticated components I built. It's a floating chatbot that uses the Groq API (after rounds of experimentation and also since the OpenAI API costs money, I switched to Groq from Hugging Face for better performance) with the LLaMA 3.3 70B model.

Key features of DermiBot include:

• Contextual conversations with message history.

• Multi-language support: responds in the user's selected language.

• Dermatology-focused responses with appropriate medical disclaimers. 

• Retry mechanisms for failed API calls. 

• Animated UI with Lottie animations and smooth transitions. 

All in all, DermiBot maintains conversation context while simultaneously enforcing strict user boundaries: it only discusses skin health topics and always recommends professional medical consultation for questions relating to confirmation of diagnosis, recommendations, etc. I thought that this was obviously necessary since the function of my Dermi app is to aid in predictions, and not to explicitly diagnose users (I don’t have the time, money, or means to fare well in the event I get sued 😬😬).

Misc. Technical Architecture: 

Navigation System

I decided to implement a dual navigation system for my app, mostly because I thought it’d look smooth and less cluttered: bottom tabs for primary screens and a pop-out drawer menu (from the left side) accessible from a menu button on all screens for secondary functions like Settings and Profile, along with other main screens including Help, Explore, Find, etc. The custom tab bar (persistent on the bottom of all screens) features a prominent center button for NewScan, and has 5 screens in total: (from L->R) Home, Explore, NewScan, History, and Find. 

State Management

Rather than using Redux or Context API, I decided to build a lightweight shared state system for managing chat history across screens and also for tracking the user gender (which prompts the switch of Male to Female model on the NewScan screen). This function also keeps the DermiBot conversations consistent regardless of where users access it (i.e. from the floating DermiBot icon across multiple screens). 

Internationalization (with i18n)

Since my grandma and grandpa only speak Chinese, and inclusivity is important, I spent the time to ensure my app supports 8 languages with over 1000 translation keys (.json files). I implemented dynamic language switching using i18n with proper font support for complex scripts like Arabic and Chinese. All UI text, including API responses, educational content, and all screens (basically every single word on the app) adapts to the selected language. Credits to Claude AI for translation since I am, unfortunately, not exactly fluent in 8 languages. 

Background Processing

The scan analysis system also handles long-running API calls gracefully. Users can start a scan, close the modal, and get notified when results are ready (the corresponding ScanResultScreen pops up immediately after API success). The system persists pending scans across app restarts and manages retry logic for failed analyses as well. 

Notification System

I built this notification system using Expo Notifications and had Dermi schedule “smart reminders” based on scan results. Malignant results trigger follow-up reminders in 14 days, while benign results get 30-day reminders (to retake a scan, just in case). Users further receive immediate notifications when scan analyses are complete.

Misc. UX Details:

Animations and Micro-interactions

• Animated splash screens were personally edited and downloaded from LottieFiles (it’s basically CapCut, but just for animations you can integrate into your code). This included a start-up splash screen for the app and animation for my DermiBot component. 

• Smooth 3D model rotation with momentum physics (in the NewScan screen). 

• Pulse animations on the DermiBot floating button.

• Loading states with progress indicators. 

• Subtle hover effects and state transitions. 

Accessibility and Inclusivity

• Support for both male and female body models (and default to male if user selects “Prefer Not to Say”). 

• Comprehensive language support including RTL languages (Arabic). 

Performance Optimizations

• Image caching and compression for scan photos. 

• Efficient 3D rendering with frame rate optimization. 

• Smart component re-rendering to prevent unnecessary updates. 

• Background processing for CPU-intensive stuff. 

Setbacks + Lessons Learned: 

The process from initial app prototype to final product was a long, arduous, and deeply rewarding process (the entire project, excluding the API set-ups, involved approximately 12,000 lines of code). 

Throughout this learning and development process, I can confidently say that I’m closer to becoming a seasoned developer. It's quite crazy to think that around a year ago, I barely knew what a loop was. While I don’t think I’d enjoy doing similar, pure software development as a full-time career, I at least now know important and relevant skills for future projects. 

Additionally, the process of having to debug features and functionalities over and over (genuinely, I spent way more time debugging than actual code-writing) also enforced within my psyche the importance of progressive advancement. Starting with a simple three-screen app and gradually adding complexity allowed me to maintain mental and workflow stability (and sanity) while slowly implementing more advanced features. However, this process was in no ways without failures, of which a major few I’ve outlined below: 

• The implementation of the bottom tab bar navigation and pop-out menu drawer was horribly disgustingly abysmally difficult. Mind you, I was creating this (and the 3D model integration, see below) in the first stage of my app prototype- when I was learning how to code concurrently with actual development. Regardless, this issue was so frustrating that I legitimately had to start over from scratch by copying and pasting my entire project line-by-line (I later figured out that I could’ve just used GitHub and it would’ve been a whole lot easier) into a new directory.

• As mentioned above, the 3D model integration was also particularly challenging: specifically, the physics of the 3D model interactions. However, paired with various tutorials and nights of relentless experimentation, I eventually did figure it out. I was so, so proud of the final product (especially since none of the apps I had reviewed in my earlier literature review stage had such a functionality). 

• Close to the end of my app development, I was plagued with every noob developer’s worst nightmare: a horrendously entangled dependency tree conflict. This was the actual bane of my existence for probably 2 months, and was further exacerbated with the release of Expo SDK 52 in November, 2024 and Expo SDK 53 in April, 2025. However, I finally resolved this conflict (which was a mesh of issues relating to react-native supported dependencies including Three.js and Lottie for web and mobile), by painstakingly reorganizing my package.json. Again, I’m sure that a more experienced developer wouldn’t have felt such despair one could only describe as comparable to that felt of Sisyphus in the Underworld, but to my amateur mind, it was actual hell on Earth (especially since my app literally stopped working for a good couple weeks). 

On a side note, building Dermi also reinforced in me the importance of appropriate disclaimers and encouraging professional medical consultation, mostly through the fear of getting sued. Thus, my app provides valuable screening tools and educational content, but always, always, ALWAYS within the context of supporting, not replacing, professional medical care. 

Regardless, beyond the few sour fruits of labor, the harvest still flourishes and the grove still stands: the final Dermi app represents around 11 months of iterative development, user feedback integration (thank you to my friends, family, and teachers, Mrs. Ament especially), and technical refinement. Overall, I've created a comprehensive platform that makes skin health monitoring accessible while still maintaining high standards for user safety and data privacy. Safe to say, this stage wraps up a large portion of my project, and I’m quite satisfied with the work I’ve accomplished and the overwhelming amount of skills and information I’ve absorbed. 

Coming soon!

Coming soon!

During Phase 2 of my project (see Very Extensive Literature Review in Dermi), I realized that current methods surrounding AI-driven skin cancer prediction, being a relatively new and developing field, has many areas in need of improvement. Specifically, I identified the following major problems:

1. In a real world setting, there exist patients of various ethnicities and consequently, skin colors. However, current CNN models are not extensively trained on non-white skin samples. Considering the relative scarcity of diverse skin lesion datasets, researchers have proposed the usage of synthetic data generation as a means to expand AI predictive abilities to all patients, regardless of skin color. 

2. Additionally, the usage of AI-driven skin cancer prediction has generally been targeted towards the general population, namely through finicky apps with preposterous paywalls or subscriptions. Many studies I read referenced the need for more substantial clinical validation in order to implement AI as a tool for dermatologists to reference and use. 

3. While I strongly believe that current AI developments in the field have the potential to be used as tools for human professionals, I began to consider the problem of cost and accessibility of dermatologist access, specifically in rural or impoverished regions across the globe. 

As a result of my findings (as well as my unquenchable thirst for glory), I decided to continue my Dermi project through 2 extension projects: 

1. The design and production of DermiScope, a low-cost, 3D-printed device-attachable dermatoscope.

2. The development of DermiVision: a novel AI vision pipeline specifically focusing on minimizing racial bias in AI skin cancer detection. 

You can read more about these 2 extension projects of Dermi in their respective project pages here: Dermiscope + DermiVision. 

Coming soon!