At Razi Title, our commitment to innovation drives us to explore and implement cutting-edge technologies. In this blog, I'll take you through our journey in developing a sophisticated text classification model using TensorFlow, delving into the advanced machine learning techniques we've employed.

import os

from layers.attention_layers import ScaledDotProductAttention

# Set environment variable to make TensorFlow use CPU
os.environ["CUDA_VISIBLE_DEVICES"] = ""

Our journey begins with preparing our machine learning environment. Choosing TensorFlow, a robust and versatile framework, allows us to build complex models with relative ease. The decision to run computations on the CPU is opportunistic, as the PC we used has a modern i9 processor which out performed the Nvidia 12GB Vram video card. Extra bonus, ensuring our model remains deployable in diverse environments without depending on specific hardware capabilities helps.

  
import os
import numpy as np
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from tensorflow.keras.layers import TextVectorization
from tensorflow.keras.layers import Dense, Dropout, Embedding, Input, Flatten
from tensorflow.keras.callbacks import EarlyStopping
import tensorflow as tf

from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.utils import to_categorical


def load_classified_data(input_directory):
    data = []
    labels = []

    for label_dir in os.listdir(input_directory):
        full_dir_path = os.path.join(input_directory, label_dir)
        if os.path.isdir(full_dir_path):
            for file_name in os.listdir(full_dir_path):
                file_path = os.path.join(full_dir_path, file_name)
                with open(file_path, "r") as file:
                    text = file.read()
                    data.append(text)
                    labels.append(label_dir)

    return data, labels

Our load_classified_data function is more than just a data loader; it's a gateway to structured machine learning. By categorizing text from various directories, we lay a solid foundation for supervised learning. This method ensures that our model can differentiate between distinct categories, which is vital in automating document classification in real estate transactions. Essentially, put text files into folders. Those folders become labels.

if __name__ == "__main__":
    input_directory = "/home/rzwink/working/cinnabar/ld/20231228094239"
    data, labels = load_classified_data(input_directory)

    NUM_WORDS = 1000
    EMBEDDING_DIM = 16
    MAX_LENGTH = 500
    # Train the model
    NUM_EPOCHS = 20
    BATCH_SIZE = 32  # You can adjust this value as needed

    vectorizer = TextVectorization(
        max_tokens=NUM_WORDS, output_sequence_length=MAX_LENGTH, output_mode="int"
    )
    vectorizer.adapt(data)

    # Save the vectorizer vocabulary
    vocab = vectorizer.get_vocabulary()
    vocab_dict = {word: index for index, word in enumerate(vocab)}

    # Encoding the labels
    label_encoder = LabelEncoder()
    encoded_labels = label_encoder.fit_transform(labels)

    # Save the label encoder
    np.save(f"models/page_classifier/label_encoder.npy", label_encoder.classes_)

    X_train, X_test, y_train, y_test = train_test_split(
        np.array([[s] for s in data]),
        to_categorical(encoded_labels),
        test_size=0.2,
        random_state=42,
    )

    # Define the model with ScaledDotProductAttention
    text_input = Input(shape=(1,), dtype=tf.string)  # Adjusted for a batch of strings
    x = vectorizer(text_input)
    x = Embedding(NUM_WORDS, EMBEDDING_DIM, input_length=MAX_LENGTH)(x)

    # Create query, key, value for attention
    query = Dense(EMBEDDING_DIM)(x)
    key = Dense(EMBEDDING_DIM)(x)
    value = Dense(EMBEDDING_DIM)(x)

In this section, we dive into the nuances of text vectorization and label encoding. Text vectorization converts raw text into numerical tokens, a format that neural networks can understand and process. This step is crucial in extracting meaningful patterns from text data, enabling the model to learn from linguistic structures. Our use of a limited vocabulary size (NUM_WORDS) and sequence length (MAX_LENGTH) balances model complexity with computational efficiency.

  
    # Apply Scaled Dot Product Attention
    attention_output, _ = ScaledDotProductAttention()(query, key, value)

    x = Flatten()(attention_output)

    x = Dense(24, activation="relu")(x)
    x = Dropout(0.5)(x)
    outputs = Dense(len(label_encoder.classes_), activation="softmax")(x)

    model = Model(text_input, outputs)

    # Compile and train the model
    model.compile(
        loss="categorical_crossentropy",
        optimizer=Adam(learning_rate=0.001),
        metrics=["accuracy"],
    )
    # Define the early stopping callback
    early_stopping = EarlyStopping(
        monitor="val_loss",  # Monitor the validation loss
        patience=3,  # Number of epochs with no improvement after which training will be stopped
        restore_best_weights=True,  # Restore model weights from the epoch with the best value of the monitored quantity
    )

    # Train the model with early stopping
    model.fit(
        X_train,
        y_train,
        epochs=NUM_EPOCHS,
        batch_size=BATCH_SIZE,
        validation_data=(X_test, y_test),
        callbacks=[early_stopping],  # Add the callback here
    )
    # Save the model
    model.save("models/page_classifier", save_format="tf")

Training a deep learning model is a delicate balance between learning enough and not overfitting. Our use of the EarlyStopping callback is a testament to our focus on creating a robust model. This approach ensures that our model retains its ability to generalize to new, unseen data, which is crucial in the dynamic field of real estate where new document types and terminologies emerge regularly.

    # Evaluate the model
    y_pred = model.predict(X_test)
    y_pred_labels = label_encoder.inverse_transform([np.argmax(y) for y in y_pred])
    y_test_labels = label_encoder.inverse_transform([np.argmax(y) for y in y_test])

    # Print classification report
    print(classification_report(y_test_labels, y_pred_labels, zero_division=1))  

The evaluation phase is where theory meets practice. By closely examining the classification report, we gain insights into the model's performance across various document types. This helps us fine-tune the model for even better accuracy in real-world applications, such as automating title searches and document verification processes at Razi Title. Through this deep dive into our machine learning model, we at Razi Title are not just embracing technological advancement; we're actively shaping it to fit our unique industry needs. This model represents a significant step in our ongoing journey to revolutionize the real estate sector with AI-driven solutions.

Greetings from the tech trenches! I'm the CTO of Razi Title, Inc., a tech startup pushing the envelope of what's possible in our industry. As any tech team knows, maintaining a clean and reliable testing environment is an essential aspect of our daily operations. The ability to quickly set up, tear down, and refresh our testing data is critical for ensuring our software's quality and reliability. Today, I wanted to share some tricks we've developed using PostgreSQL stored procedures to facilitate this process.

In PostgreSQL, a stored procedure allows us to encapsulate and store complex SQL queries for later execution. In our case, we have created two stored procedures - `truncate_tables` and `copy_data` - that, when used in tandem, ensure we have a fresh and reliable testing environment ready for action.

Truncating Tables: `truncate_tables`

Our first stored procedure, `truncate_tables`, is a workhorse in our test data management toolkit. It loops through all tables in a given schema and truncates them, effectively wiping all data and preparing for a fresh test run.

Here is the code:

CREATE OR REPLACE PROCEDURE truncate_tables(schema_to text) AS
$$
DECLARE
    _tbl text;
BEGIN
    FOR _tbl IN
        SELECT tablename FROM pg_tables WHERE schemaname = schema_to
    LOOP
        EXECUTE format('TRUNCATE TABLE %I.%I CASCADE', schema_to, _tbl);
    END LOOP;
END;
$$ LANGUAGE plpgsql;

Once the old test data is purged, we use our second stored procedure, `copy_data`, to copy fresh data from our source schema to the test schema. This procedure is unique and somewhat nuanced, as it takes into account that the column order in the source and target tables might not be identical.

This difference in column order can occur even when both schemas have been generated by Django's `migrate` command, as Django does not enforce a specific column order. But fear not, our stored procedure handles this gracefully by explicitly specifying column names in both the `INSERT INTO` and `SELECT` statements. If a table does not exist in the target schema, it is simply skipped.

Here is the `copy_data` code:

CREATE OR REPLACE PROCEDURE copy_data(schema_from text, schema_to text) AS
$$
DECLARE
    _tbl text;
    _exists boolean;
    table_arr text[];
    i int;
BEGIN
    -- Recursive CTE to find tables in order of foreign key dependency
    WITH RECURSIVE fk_tables AS (
        -- Find tables that have no foreign keys into them (base case)
        SELECT
            tb.oid,
            tb.relname AS tablename,
            array_agg(tb.relname) AS all_tables
        FROM
            pg_class AS tb
            LEFT JOIN pg_constraint AS fk ON fk.confrelid = tb.oid
        WHERE
            tb.relkind = 'r'
            AND fk.oid IS NULL
            AND tb.relnamespace = (SELECT oid FROM pg_namespace WHERE nspname = schema_from)
        GROUP BY
            tb.oid,
            tb.relname
        UNION ALL
        -- Recursively find tables that have a foreign key into a table we've seen
        SELECT
            tb.oid,
            tb.relname AS tablename,
            all_tables || tb.relname
        FROM
            fk_tables
            JOIN pg_constraint AS fk ON fk.conrelid = fk_tables.oid
            JOIN pg_class AS tb ON fk.confrelid = tb.oid
        WHERE
            NOT tb.relname = ANY(all_tables)
    )
    SELECT array_agg(tablename) INTO table_arr FROM fk_tables;

    FOR i IN 1 .. array_length(table_arr, 1)
    LOOP
        _tbl := table_arr[i];
        EXECUTE format('SELECT EXISTS (SELECT 1 FROM pg_tables WHERE schemaname = %L AND tablename = %L)',
                       schema_to, _tbl) INTO _exists;
        IF _exists THEN
            EXECUTE (
                SELECT
                    'INSERT INTO ' || schema_to || '.' || _tbl ||
                    ' ("' || string_agg(column_name, '", "') || '") SELECT "' ||
                    string_agg(column_name, '", "') || '" FROM ' || schema_from || '.' || _tbl || ';'
                FROM
                    information_schema.columns
                WHERE
                    table_schema = schema_from AND table_name = _tbl
                GROUP BY
                    table_schema,
                    table_name
            );
        END IF;
    END LOOP;
END;
$$ LANGUAGE plpgsql;

These two procedures have become instrumental for us to quickly and effectively manage our test environment. For instance, we can schedule them to run before every major test suite, ensuring the latest data structure and content are in place, mimicking our production environment as closely as possible.

I hope that by sharing our approach, other teams may find inspiration or at least a starting point to address similar challenges in their development process. Happy testing!

Stay tuned for more insights from the front lines of startup tech. If you have any questions or thoughts, feel free to share them in the comments below!

Best,

Rob Zwink

CTO, Razi Title, Inc.

In the dynamic real estate industry, the ability to adapt and innovate is critical. As the CTO of a leading software solutions provider, I've seen how targeted technology can help our customers — real estate and title companies — navigate this complex landscape.

Central to our approach is the development of highly customized applications, designed to meet the specific needs of each customer. Customized software solutions provide the flexibility that our customers need to design and implement processes that align perfectly with their unique business models. When it comes to creating these applications, one tool rises to the top: Form.io.

Form.io is a focused platform that simplifies the process of creating complex forms and managing data. It's an ideal solution for our customers, particularly title companies, as it enables seamless data collection and transfer, ensuring they always have accurate and up-to-date information.

One key advantage of Form.io is its ability to handle large volumes of data. Many of our customers, like title companies, handle an enormous amount of data daily. Form.io empowers them to manage this data effectively, allowing for informed decision-making based on real-time, accurate information.

Additionally, Form.io's server-less architecture greatly reduces the complexity of their software stack. This leads not only to cost savings but also to an increase in the reliability and maintainability of their applications.

Form.io's capabilities extend beyond data management. It's also an effective tool for crafting interactive user interfaces. Using Form.io, we've been able to design intuitive interfaces for our customers' applications, enhancing the user experience in their services.

The real value of our services comes from how we combine our deep understanding of the industry, our customers, and technology. We leverage these insights to develop highly customized applications that truly add value.  Essentially, anyone with terms like 'title, LLC' an there name would benefit when building custom applications.

The success we've achieved with Form.io showcases the potential of combining industry understanding with advanced technology. By harnessing Form.io, we create highly customized applications that drive efficiency, enhance customer satisfaction, and contribute to the success of our customers in the real estate industry.

Our journey with Form.io goes beyond just optimizing highly customized applications. It's about leveraging technology to help our customers deliver superior service, drive growth, and shape the future of real estate. We are excited about the endless possibilities that lie in store.

Hello to all of our dedicated real estate professionals,

As the Chief Technology Officer of Razi, I often work with various stakeholders within the real estate industry. And what I consistently find is that we all face a common struggle: dealing with problematic PDF files. Now, before you dismiss this as a minor issue, consider the critical role that PDF files play in real estate transactions. These documents contain contracts, property details, and critical customer information. A problem with a PDF can delay a deal, create confusion, or even lead to losses.

While there are numerous tools available to address these challenges, the most common choice in the industry has been Ghostscript. Don't get me wrong; Ghostscript has been an instrumental tool in handling PDFs for many years. However, we've found that it has its limitations, particularly when it comes to repairing damaged or problematic PDF files. This can be problematic for real estate title offices, where precision and accuracy are paramount.

Enter Mutool, a hidden gem in the world of PDF repair.

Mutool, an open-source tool created by the team behind the MuPDF library, is a tool designed specifically for dealing with all types of PDFs. The power of Mutool is that it repairs most PDFs, even those that Ghostscript fails to fix. This is a tool in the toolbelt for title insurance and title offices, where one problematic PDF can cause delays and unnecessary headaches.

Here's how it works:

• Identify and Repair Corrupted Files: Mutool works by identifying any corrupted elements within a PDF file, even in those cases where the corruption may not be immediately obvious. Once identified, Mutool will attempt to repair the file by reconstructing the corrupted elements as much as possible.
• Preserve Original Document Integrity: In the real estate industry, it's essential to maintain the integrity of documents. Mutool understands this, and its repair process is designed to preserve as much of the original document as possible, ensuring that all critical data is retained.
• Comprehensive and Robust: Unlike Ghostscript, Mutool is robust and comprehensive in its approach to handling PDF files. It's able to tackle more complex corruption issues, offering a higher success rate in repairs.
• User-Friendly: Mutool was designed with the user in mind. Its simplicity and easy-to-use interface make it an ideal tool for any title office, regardless of technical proficiency.

In the ever-evolving landscape of real estate transactions, it's crucial to adapt and employ tools that streamline our operations. Mutool represents a step forward in handling PDF issues, ensuring we can effectively manage our documents and deliver exceptional service to our clients.

As a tech leader in the real estate space, I'm always excited to share these innovations. By integrating tools like Mutool, we can address common pain points, improve our efficiency, and ultimately, enhance the client experience. I hope this post has shed some light on the benefits of Mutool, and I encourage you to explore how it can improve your operations.

Till our next tech discussion, keep innovating and transforming!

Stay tuned!

Rob Zwink

CTO at Razi 

Introducing our newest addition to the Razi Exchange team: Robert Zwink, our esteemed Chief Technology Officer (CTO). With an impressive background spanning over years at JP Morgan Chase, co-founding a previous start-up, serving as CTO at an acquired fintech venture, and most recently as Senior Vice President of Engineering at a prominent fintech establishment, Robert brings a wealth of expertise to the table. Renowned for his innovative approach to new technologies, his uncanny ability to conquer challenging architectures, and his foresight in identifying market needs ahead of the curve, Robert is set to lead our technological endeavors with unwavering determination and a pinch of tech-savvy charm.

Welcome aboard, Robert – time to strengthen the community!

As ChatGPT and similar artificial intelligence (AI) models gain popularity, title professionals from Washington, D.C.-based Razi Title, Inc. are finding ways to use those innovations to improve everyday efficiency. Razi Title Chief Technology Officer Robert Zwink and Chief Executive Officer Lili Farhandi sat down with The Title Report to discuss what they’ve dubbed “TitleGPT,” a conversational AI built in-house that specializes in title industry-related documentation.

“We didn’t build this from total scratch,” Zwink said. “We’ve made use of the same technology that ChatGPT uses. Ours is unique, standalone and purpose-built for indexing title documents. When I first started interacting with ChatGPT, I asked if there was a word for the anxiety someone can feel in harnessing this type of technology. “Lili and I are very passionate about strengthening the title community. At the end of the day, we want to put this at the fingertips of all title professionals, distill this tech down in a way everyone can understand, and reduce the time it takes to get to quality results.”

TitleGPT is already being implemented by a handful of select companies and users of Razi Exchange, a Razi Title, Inc. community that lets professionals exchange prior policies, create efficiency in the title search, shorten the time to close and unlock new business.

 Zwink and Farhandi said they plan to oversee wider distribution by the middle of 2023, starting off with a free access model. Farhandi went over ways conversational AIs can push the title industry forward over the next five to 10 years.

“I’d love to see this technology go somewhere where we have accurate data on the title and not doing what I call the ‘rework’ in putting title insurance documents together,” Farhandi said. “This is a real opportunity to remove a lot of redundancies and make everyday duties that much easier for everyone involved.” Zwink said it’s too early to quantify exactly how much time is saved by using TitleGPT for indexing documents as opposed to performing those tasks manually, but he said the benefits are clear as day. “When you think about the number of hours saved from having a reliable AI available for this indexing work, it’s really hard to count that,” he said. “ChatGPT provides the answer to questions in a very modern way, and to be able to do that reliably at a fraction of the cost is very beneficial.

“What Razi has done has all been in-house. We don’t rely on ChatGPT as an external service. That’s very exciting. Not only can we lend efficiency, but we can lend it at scale without driving up the cost. Our goal is to make this technology accessible and affordable.”

Experts have cited several ways ChatGPT-related tech can benefit title and other insurance sectors in the short term and long term, including underwriting, customer support, claims processing and fraud protection. ChatGPT can be trained to assist in the detection of insurance fraud, where it can identify patterns of inconsistencies and flag them for human review. According to the FBI, the total cost of insurance fraud, not counting health insurance cases, is estimated to be more than $40 billion per year.

“It’s exciting to think about a more efficient and less costly way to perform a title search with the peace of mind on the accuracy of your data” Farhandi said. “We’re very excited thinking about what comes next and new ways we can help the title community, to put it right in front of them so it’s there whenever they need it.” Multiple cybersecurity firms offering services to title agencies have recently said instances of fraud and attempted fraud remain at all-time highs.

Proponents of ChatGPT and similar tech also stress it should be used for human support rather than a total replacement for manual work or existing technology. “There’s been a continued investment put into quality control with (AI tech) and that’s not going to go away,” Zwink said. “Then you have the confluence of humans and computers and how they work together. It’s something we’ll keep seeing, a continued balance in the force. It’s going to remain very, very important.

Razi Exchange launched a national independent prior policy technology platform. The company said it aimed to enable a more efficient title transfer process by creating an open market exchange to buy and sell prior policies, organize and search existing policies, and generate business leads.

“Today, title professionals create valuable intellectual property that is inefficient to leverage to its maximum potential,” Razi Exchange co-founder and CEO Lili Farhandi said in a release. “We have listened to real-world challenges from leaders in the industry and have created the first independent national platform. The Razi Exchange helps lower the costs and the time of performing a title search, resulting in a more predictable closing, all the while enabling platform users to unlock new revenue streams.

“Customers who use the Razi Exchange platform save time on the closing process by not requiring a full title search on each transaction, shortening the title search process by an average of approximately 80 percent,” she added. According to Farhandi, Razi has thousands of policies on the platform, the count is growing daily.