A strange problem with modeless dialog boxes

[Alex Sokolek]

I have solved the problem! When the AI Bot told me that I was stalling the message loop, I looked deeper. In the original code I was calling PeekMessage() with PM_REMOVE, filtering for WM_KEYDOWN, and then checking for VK_ESCAPE. This consumed the WM_KEYDOWN message but not any other messages. Now, I am calling PeekMessage() with PM_REMOVE, with no filtering, and then checking for WM_KEYDOWN and VK_ESCAPE. The result is that I am consuming all messages, and detecting WM_KEYDOWN/VK_ESCAPE.

// Check for ESC pressed - Abort if so.
MSG msg;
if (!PeekMessage(&msg, NULL, WM_KEYDOWN, WM_KEYDOWN, PM_REMOVE)) continue;
if (msg.wParam != VK_ESCAPE) continue;
pCHashedFiles->Reset();
bEscapePressed = true;
break;

becomes...

// Check for ESC pressed - Abort if so.
MSG msg;
if (!PeekMessage(&msg, NULL, 0, 0, PM_REMOVE)) continue;
if (msg.message != WM_KEYDOWN || msg.wParam != VK_ESCAPE) continue;
pCHashedFiles->Reset();
bEscapePressed = true;
break;

Now the program works properly, in both debug executions and in non-debug executions.

I still don't know why it worked in debug mode, but I'm not "going to look a gift-horse in the mouth".

I have left the original post intact, for reference...

Hi. I'm having a problem with modeless dialog boxes. I have one in a loop that scans files in a directory, adding the files to a linked list class for later processing. This used to work just fine. Now, without making any changes, it freezes part way through and stops updating. I know the loop is still running, because after a period of time (the same time it used to take) the loop terminates and I see the results. Sometimes the point of freeze moves around, but ususally it happens at the same spot.

The strange part of this is that it only occurs with a Release build. A Debug build works just fine.

This has all the hallmarks of uninitialized memory and/or buffer overrun, but for the life of me I can't find it. I have spent the last three days instrumenting the loop, trying to find evidence of where the problem is, but no help.

Anyway, here is the code for the command handler. I apologize for the length of this posting, but I ahve no idea where I'm going wrong, so I thought it best to post the whole routine. The dialog box gets updated in the section "Update the user about progress". Please take a look and see what you can see. Thank you.

        case ID_FILE_SCAN:
            /////////////////////////////////////////////////////////////////////////////////////////////////
            // Select directory and then scan, hash, and sort.
            /////////////////////////////////////////////////////////////////////////////////////////////////
            {
                MessageBox(hWnd, _T("Navigate to the desired directory and double\n"
                                    "click on any file to process the directory."), szTitle, MB_OK);

                OPENFILENAME ofn;
                ZeroMemory(&ofn, sizeof(ofn));

                // Initially, _T(""), replaced by the path to scan via ofn.lpstrFile
                TCHAR* pszOpenFileName = new TCHAR[MAX_PATH];
                ZeroMemory(pszOpenFileName, sizeof(pszOpenFileName));

                ofn.lStructSize = sizeof(OPENFILENAME);
                ofn.hwndOwner = hWnd;
                ofn.lpstrFile = pszOpenFileName;
                ofn.nMaxFile = MAX_PATH;

                // Save original current directory.
                GetCurrentDirectory(MAX_PATH, szOldDirectoryName);

                int LastAPICallLine = __LINE__ + 1;
                if (!GetOpenFileName(&ofn)) // Retrieves the fully qualified file name that was selected.
                {
                    if (CommDlgExtendedError() == 0)
                    {
                        delete[] pszOpenFileName;
                        break;
                    }
                    TCHAR sz[MAX_ERROR_MESSAGE_LEN];
                    StringCchPrintf(sz, MAX_ERROR_MESSAGE_LEN,
                        _T("API Error occurred at line %ld error code %ld"), LastAPICallLine, CommDlgExtendedError());
                    MessageBox(hWnd, sz, _T("ApplicationRegistry.cpp"), MB_OK + MB_ICONSTOP);
                    break;
                }
                ofn.lpstrFile[ofn.nFileOffset] = TCHAR('\0'); // We only want the path, so eliminate the file name.
                WIN32_FIND_DATA Win32FindData;

                FILETIME LocalFileTime;
                SYSTEMTIME LocalSystemTime;

                #define FORMATTED_FILE_DATE_LEN 11
                TCHAR* pszFileDate = new TCHAR[FORMATTED_FILE_DATE_LEN];

                #define FORMATTED_FILE_TIME_LEN 6
                TCHAR* pszFileTime = new TCHAR[FORMATTED_FILE_TIME_LEN];

                #define FORMATTED_FILE_SIZE_LEN 21
                TCHAR* pszFileSize = new TCHAR[FORMATTED_FILE_SIZE_LEN];

                pCHashedFiles->Reset();

                // Save directory name in global array for use by other routines
                StringCchCopy(szDirectoryName, MAX_PATH, ofn.lpstrFile);
                SetCurrentDirectory(ofn.lpstrFile);

                // Count the files in the directory.
                int iTotalFiles = 0;
                HANDLE hFind = FindFirstFile(_T(".\\*.*"), &Win32FindData);
                if (hFind == INVALID_HANDLE_VALUE) break;
                do iTotalFiles += Win32FindData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY ? 0 : 1;
                while (FindNextFile(hFind, &Win32FindData) != 0);
                FindClose(hFind);

                // Setup to use the modeless dialog box to display progress.
                dc = GetDC(hWndProgressBox);
                TCHAR szFilesProcessed[200];
                RECT WindowRect;
                GetWindowRect(hWnd, &WindowRect);
                ShowWindow(hWndProgressBox, SW_SHOW);
                SetWindowPos(hWndProgressBox, HWND_NOTOPMOST, WindowRect.left+50, WindowRect.top+50, 0, 0, SWP_NOSIZE | SWP_SHOWWINDOW);
                uint64_t FileSize;

                // Get the first file in the directory
                hFind = FindFirstFile(_T(".\\*.*"), &Win32FindData);
                if (hFind == INVALID_HANDLE_VALUE)
                {
                    ShowWindow(hWndProgressBox, SW_HIDE);
                    delete[] pszFileDate;
                    delete[] pszFileTime;
                    delete[] pszFileSize;
                    delete[] pszOpenFileName;
                    break;
                }

                BOOL bEscapePressed = false;
                BytesProcessed = 0;

                // Process this and all of the rest of the files in the deirectory.
                do
                {
                    // Ignore subdirectories.
                    if (Win32FindData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) continue;

                    // Get the last write time and format the local time equivalent of it.
                    FileTimeToLocalFileTime(&Win32FindData.ftLastWriteTime, &LocalFileTime);
                    FileTimeToSystemTime(&LocalFileTime, &LocalSystemTime);
                    StringCchPrintf(pszFileDate, FORMATTED_FILE_DATE_LEN, _T("%02d/%02d/%04d"),
                                    LocalSystemTime.wMonth, LocalSystemTime.wDay, LocalSystemTime.wYear);
                    StringCchPrintf(pszFileTime, FORMATTED_FILE_TIME_LEN, _T("%02d:%02d"),
                                    LocalSystemTime.wHour, LocalSystemTime.wMinute);

                    // Get the file size and format it.
                    FileSize = (uint64_t)Win32FindData.nFileSizeHigh * ((uint64_t)MAXDWORD + 1) +
                               (uint64_t)Win32FindData.nFileSizeLow;
                    StringCchPrintf(pszFileSize, FORMATTED_FILE_SIZE_LEN, _T("%9llu"), FileSize);
                    BytesProcessed += FileSize;

                    // Generate sha1 digest.
                    SHAFile.Process(Win32FindData.cFileName, 0, pszMessageDigest, NULL);

                    // Add the file information to the HashedFiles class.
                    pCHashedFiles->AddNode(pszMessageDigest, pszFileDate, pszFileTime, pszFileSize, Win32FindData.cFileName);
                  
                    // Update the user about progress.
                    int iPercent = (int)((float)pCHashedFiles->GetNodeCount() * 100.f / iTotalFiles + 0.5f);
                    StringCchPrintf(szFilesProcessed, 200,
                                    _T("Files processed: %u     %d%% of %d     MBytes processed: %llu"),
                                    pCHashedFiles->GetNodeCount(), iPercent, iTotalFiles, BytesProcessed/1024/1024);
                    SetBkColor(dc, RGB(240, 240, 240));
                    TextOut(dc, 16, 16, szFilesProcessed, lstrlen(szFilesProcessed));
                    TextOut(dc, 16, 40, _T("Press ESC to abort."), 19);

                    // Check for ESC pressed - Abort if so.
                    MSG msg;
                    if (!PeekMessage(&msg, NULL, WM_KEYDOWN, WM_KEYDOWN, PM_REMOVE)) continue;
                    if (msg.wParam != VK_ESCAPE) continue;
                    pCHashedFiles->Reset();
                    bEscapePressed = true;
                    break;

                } while (FindNextFile(hFind, &Win32FindData) != 0);

                MessageBeep(MB_ICONASTERISK);
                FindClose(hFind);
                ReleaseDC(hWndProgressBox, dc);

                // Sort by hash then file.
                iSortMode = 0;
                if (!bEscapePressed) pCHashedFiles->SortAndCheck(iSortMode);
              
                // Setup initial view.
                bMarked = false;
                iStartNode = 0;
                iSelectedFile = 0;
                InvalidateRect(hWnd, NULL, true); // Generate paint message.

                delete[] pszFileDate;
                delete[] pszFileTime;
                delete[] pszFileSize;
                delete[] pszOpenFileName;

                // Restore original current directory.
                SetCurrentDirectory(szOldDirectoryName);

                // Set a 3 second timer to close the modeless dialog box.
                if (!bEscapePressed) uiTimer = SetTimer(hWnd, 1, 3000, NULL);
                else                 uiTimer = SetTimer(hWnd, 1, 30,   NULL);
            }
        break;

 

[ChatGPT]

From your description of the problem and examining the code you posted, it does seem reasonable to guess at memory issues as a possible cause. You mentioned that the problem occurs in Release mode but not Debug mode, which often indicates a memory bug. This is because Debug and Release builds use different memory allocation strategies. Debug mode often pads uninitialized memory differently, which can unintentionally mask a memory bug that becomes fatal in Release mode when your software is confronted with the raw hardware behavior.

Looking at your code, there is a glaring issue with the usage of `sizeof(pszOpenFileName)` in your `ZeroMemory` call.

Here is the code portion in question:

```cpp
// Initially, _T(""), replaced by the path to scan via ofn.lpstrFile
TCHAR* pszOpenFileName = new TCHAR[MAX_PATH];
ZeroMemory(pszOpenFileName, sizeof(pszOpenFileName));
```
You are calling `sizeof()` on a pointer (`pszOpenFileName`), which will return the size of the pointer itself (usually 4 or 8 bytes depending on your system), not the size of what it points to. The effect of this is that most of your memory block will not get zeroed out as expected.

Your call to `ZeroMemory` should look like this:

```cpp
ZeroMemory(pszOpenFileName, sizeof(TCHAR) * MAX_PATH);
```
This zeroes out the entire buffer as expected.

Another point of concern might be how you are consuming Windows messages for checking the escape key:

```cpp
MSG msg;
if (!PeekMessage(&msg, NULL, WM_KEYDOWN, WM_KEYDOWN, PM_REMOVE)) continue;
if (msg.wParam != VK_ESCAPE) continue;
pCHashedFiles->Reset();
bEscapePressed = true;
break;
```
You are only consuming `WM_KEYDOWN` messages and ignoring the rest. It is possible that your message queue is filling up with unprocessed messages, leading to unexpected issues.

Furthermore, you might want to consider using a tool that can help you to identify memory issues such as uninitialized memory or buffer overruns. Valgrind is such a tool on Unix systems, for Windows Visual Studio has a built-in memory debugger that you might find useful.

I hope you find these hints helpful. If the problem persists afterwards, don't hesitate to ask further questions.

 

[Alex Sokolek]

I will try to do the ZeroMemory as you suggested and get back to this forum when I have results. I don't think the problem is ESC processing as I am not pressing ESC during execution of the loop. Thank you.

 

[Alex Sokolek]

I replaced the call to ZeroMemory with "ZeroMemory(pszOpenFileName, MAX_PATH * sizeof(TCHAR));'. No help.

How do I use the VS Memory Debugger? Thanks.

 

[Alex Sokolek]

An update, after several hours of testing...

1.) It does not matter if it is a Release build or a Debug build. (I was in error.) What matters is how the app is launched. If I launch with <Ctrl>F5 (Run without debugger) the odd behavior is manifest. If I Launch with F5 (Run with debugger) the odd behavior does not manifest. Both Release and Debug builds exhibit the same behavior.

2.) I commented out all of the code except the dialog box code itself and the problem still persists. This makes me think the problem is with the dialog box code itself.

3.) I have noted a second symptom... When the loop is done, I start a three second timer. I don't hide the dialog box until the timer expires. This gives the user a chance to see the results after the dialog box stops moving. When I run without debugging, the dialog box blanks out at the end of the loop and then disappears after three seconds. It does not matter if the dialog box "freezes up" while counting up. I tried this with a second directory with fewer files... This way the "freezing" does not occur, but the blanking out does. (That is, without the debugger.)

I'm pretty sure this has something to do with how I create, show, and/or paint the dialog box. Note that I am not using a dialog box procedure. It is supposed to be optional. I paint in the dialog box with a DC obtained from GetDC(). Supposedly, the dialog box manager handles all of the dialogbox messages, similar to DefWindowsProc().

Please take another look at the code, concentrating on the dialog box code. I remain at a loss to explain this. Thank you.

 

[Alex Sokolek]

I noted something else... When the dialog box freezes, the mouse cursor changes from an arrrow to an hourglass and the "X" button in the upper right corner of the parent window changes to dim red. Interesting, eh...

 

[Alex Sokolek]

Yet another observation... It takes five seconds for the dialog box to freeze and the cursor and "X" button to change. It does not matter how many files are in the directory to scan, or how much delay I add to each iteration of the loop - its always five seconds.

I'm thinking that Windows is timing out waiting for something to happen, but it never happens. I tried creating a dummy dialog box procedure which returns false. That did not help. (Returning true, however, results in a dead application that has to be terminated with Task Manager.)