R14. Stress Testing AI Models for Indian SME Credit Risk

1. Research Problem

How do credit-risk models behave under economic stress scenarios and missing informal-economy signals?

This research project is intended for a university research group, not for a short classroom demonstration. The goal is to investigate an unanswered or partially answered question in Finance AI. The research group should begin with the known baseline project, Credit Risk Scoring Portal, and then move beyond implementation into systematic experimentation. The result may be positive, negative, or inconclusive; all three outcomes can be valuable if the methodology is rigorous and the evidence is clear.

The core problem is that current AI systems often work well in controlled demonstrations but fail when data is noisy, domain-specific, low-resource, multilingual, biased, incomplete, or operationally constrained. This research topic asks the group to identify one such limitation, formulate a research hypothesis, and test it through reproducible experiments. The project should not claim novelty simply because it uses a modern model. Novelty may come from a new dataset, a careful benchmark, a domain adaptation study, a failure taxonomy, an evaluation framework, an efficiency improvement, or a reproducibility result.

A strong research outcome should explain what was already known, what gap remained, what was attempted, what evidence was collected, and what future researchers should do next. The group should maintain experiment logs from the beginning because research quality depends not only on the final result but also on the traceability of decisions, failed runs, assumptions, and parameter choices.

2. Research Background and Gap

The baseline project Credit Risk Scoring Portal provides an implementation starting point. It gives the research group a predictable system to reproduce before attempting any new contribution. Reproducing the baseline is important because it prevents the group from confusing implementation errors with research findings. Only after the baseline works should the group introduce a new method, dataset, benchmark, or evaluation strategy.

The research gap is to go beyond a working prototype and ask whether a measurable improvement, new insight, or new failure analysis can be produced. Suggested technical directions include scenario simulation, fairness analysis, explainability. The group should compare these directions with recent literature and decide which one is realistic for the available time, compute, data access, and mentor expertise.

The broader publication direction is Responsible FinTech. A publishable paper usually needs more than a system screenshot. It needs a clear research question, related work, reproducible method, experimental comparison, results table, error analysis, limitations, and discussion of why the findings matter. If the result is negative, the group can still produce a useful paper if it explains why the expected improvement did not occur and what that reveals about the problem.

The work should be planned as a continuation-friendly research stream. One student batch may reproduce the baseline and create the benchmark. The next batch may test improved methods. A later batch may extend the dataset, add stronger evaluation, or prepare the final paper. This continuity is important for university research groups because serious research often requires more time than a single semester.

Why this is research and not only development

A development project proves that a system can be built. A research project investigates an unanswered question. This project should produce new evidence: a benchmark, comparison, failure taxonomy, method improvement, dataset extension, domain adaptation study, reproducibility result, or evaluation framework that other researchers can build on.

4. Research Questions

• What limitation exists in the baseline approach?
• What new method, dataset, benchmark, or evaluation idea will the group test?
• How will the group prove improvement or identify failure?
• What negative result would still be useful to future researchers?
• Which result would be strong enough for a Scopus-indexed conference or journal submission?

5. Literature Review Direction

The group should read at least 15-25 papers before finalizing the method. The literature review should not be a generic summary. It should compare datasets, methods, evaluation protocols, limitations, and reproducibility gaps.

• Find 5 recent survey or benchmark papers in this domain.
• Find 5 method papers related to the proposed technique.
• Find 3-5 papers that report failure cases, limitations, or negative results.
• Create a comparison table with columns for dataset, method, metric, result, limitation, and code availability.

6. Suggested Methodology

• Starting baseline: Credit Risk Scoring Portal
• Suggested methods: scenario simulation, fairness analysis, explainability
• Build a reproducible baseline first and record its metrics.
• Run at least three controlled variations of the proposed research idea.
• Compare against the baseline using statistical, qualitative, and error-case analysis.
• Maintain experiment logs, configuration files, and result tables from the first week.

6A. Topic-Specific Reading Notes

Before implementation, the group should use these notes to decide what must be understood, reproduced, and challenged.

• Create a fixed question set before tuning retrieval.
• Track source document, page number, chunk ID, and retrieved passage for every answer.
• Compare answers with retrieval disabled and retrieval enabled.
• Pay attention to class imbalance and false negatives.
• Use precision-recall curves instead of accuracy alone.
• Explain risk outputs as decision support, not final approval or rejection.

7. Experimental Matrix

8. Evaluation Plan

• Define baseline metrics before proposing improvements.
• Use ablation studies to show which component helped.
• Report failure cases and negative results.
• Compare with at least two relevant methods from literature where possible.
• Use significance testing or repeated runs where randomness affects results.
• Release code, configuration, and dataset documentation if licensing allows.

9. Expected Research Output

• Research proposal and literature review.
• Baseline implementation and reproducibility report.
• Experiment table with all runs and parameters.
• Failure taxonomy or qualitative analysis section.
• Draft paper with abstract, introduction, related work, method, experiments, results, limitations, and references.
• Continuation note for the next student group.

10. Publication Direction

Potential contribution area: Responsible FinTech. The research group should target a workshop, Scopus-indexed conference, or journal only after the baseline and experiment results are strong enough.

Possible contribution types include a new dataset, a benchmark, an improved method, a careful reproducibility study, a domain adaptation result, an evaluation framework, or a failure analysis that other researchers can build on.

11. Continuation Path

• Phase 1: reproduce baseline and document gaps.
• Phase 2: test the first research hypothesis.
• Phase 3: create benchmark, dataset extension, or improved method.
• Phase 4: next batch extends the best result or studies the strongest failure case.
• Phase 5: convert stable results into a paper submission and release package.

12. References

• Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• RAGAS: Automated Evaluation of Retrieval Augmented Generation
• BEIR Benchmark for Information Retrieval
• Lopez de Prado, Advances in Financial Machine Learning
• Kaggle Credit Card Fraud Dataset
• yfinance Documentation
• SHAP Documentation
• LlamaIndex Documentation
• Google Scholar
• arXiv
• Papers With Code
• SCImago Journal and Country Rank