Using filenames with spaces in variable functions

[FoxWolfie Galen]

I'm having a problem with processing some files. The filenames are not
created by me, so I have no control over what they will be.

I am using Win 98se with 4DOS 8.00.
I have Win95SFNSearch set to no, and prefer to leave it that way.

The problem is when there are spaces in the filenames is wish to process.
How would I cause these two lines to set the filedate and filetime to
variables, as I intend, assuming there are spaces in the name?

set fd=%@filedate[%fl,,s]
set ft=%@filetime[%fl,,s]

Try a name like "m 100 ani frame 02.gif"
I tried dozens of variations, and get errors similar to the following:

D:\PICS\GI.BAT [31] File not found "C:\AnimWork\m"
set fd=
D:\PICS\GI.BAT [31] File not found "C:\AnimWork\100"
set fd=
D:\PICS\GI.BAT [31] File not found "C:\AnimWork\ani"
set fd=
D:\PICS\GI.BAT [31] File not found "C:\AnimWork\frame"
set fd=
D:\PICS\GI.BAT [31] File not found "C:\AnimWork\02.gif"
set fd=
D:\PICS\GI.BAT [32] File not found "C:\AnimWork\m"
set ft=
D:\PICS\GI.BAT [32] File not found "C:\AnimWork\100"
set ft=
D:\PICS\GI.BAT [32] File not found "C:\AnimWork\ani"
set ft=
D:\PICS\GI.BAT [32] File not found "C:\AnimWork\frame"
set ft=
D:\PICS\GI.BAT [32] File not found "C:\AnimWork\02.gif"
set ft=

Here are some of what I tried:

set fd=%@filedate["%fl",,s]
set ft=%@filetime["%fl",,s]

I thought the quotes would be enough to force the name to be a single
entity.

set fd=%@filedate[".\%fl",,s]
set ft=%@filetime[".\%fl",,s]

set fd=%@filedate["%@filename[%fl]",,s]
set ft=%@filetime["%@filename[%fl]",,s]

set fd=%@filedate[%@filename[%@sfn[%fl]],,s]
set ft=%@filetime[%@filename[%@sfn[%fl]],,s]

This last one held promise, as it gave the following errors:

D:\PICS\GI.BAT [31] File not found "C:\AnimWork\M100AN~2.GIF"
set fd=
D:\PICS\GI.BAT [32] File not found "C:\AnimWork\M100AN~2.GIF"
set ft=

Doing a "dir /nx" revealed that the file "M100AN~2.GIF" does exist, and is
the SFN of "m 100 ani frame 02.gif", in this case. Temporarily restarting
with Win95SFNSearch enabled, allows this to work, but I really don't want
to run with that on, due to other issues it causes.

What else can I do to get filenames containing spaces to work with the
internal variable functions? Is there a way to temporarily toggle
Win95SFNSearch from within a batch file? I'd settle for something like
that, so long as I could always be sure it was off outside the context of
the batch file. I tried using ' ' quotes instead of " " quotes. They
caused the long filename to be treated as one unit, but the quotes became
part of the name, and therefore caused it to fail. I also tried enclosing
the filenames in [ ] characters, with no luck.

[Klaus Meinhard]

Post by FoxWolfie Galen
The problem is when there are spaces in the filenames is wish to
process. How would I cause these two lines to set the filedate and
filetime to variables, as I intend, assuming there are spaces in the
name?

The way to go is to put double quotes around the filename, like you did

Post by FoxWolfie Galen
I thought the quotes would be enough to force the name to be a single
entity.

And what happened? Please use setdos /y1 to debug and check how the
variable %fl gets expanded.

(I am under Vista 64 at the moment, which won't run 4dos, so I cannot
test myself).

[FoxWolfie Galen]

Post by Klaus Meinhard
The way to go is to put double quotes around the filename, like you did

Post by FoxWolfie Galen
I thought the quotes would be enough to force the name to be a single
entity.

And what happened? Please use setdos /y1 to debug and check how the
variable %fl gets expanded.

With the double quotes, it was still splitting the filenames at every
space. The variable contained the full filename, up to the filedate line.
At that point, it was suddenly broken. I ultimately found the problem
though. Earlier in the batch file, I had
setdos /x-1245789
Changing that to
setdos /x-124589
was enough to make it work. All this did was shift the problem to
something a lot less common. Since the filenames I wish to process are not
my own, they can contain anything that is legal for the OS. Spaces occur
in perhaps 10 percent of the image files I process, so that needed to be
fixed. I will still likely have problems when a filename contains the
back-quote ` or the square brackets [ ]. Those cases are far less common
than spaces, so my change is an improvement. Ideally, I could process any
filename without having to go back and check that something didn't work.
I'm waiting for the day when I run into a file named:

'As-bad_as+it~gets!`, @=#$)(}{][;%^&.gif

This filename works in Win 98se and in MS-DOS, as well as most, but not all
applications. It's a difficult one to process in 4DOS though. By
processing, I mean passing such problem names to subroutines. using
variable functions with them, renaming, deleting, copying, moving, etc. I
understand that some characters are just going to be hard to work with in
an automated fashion, such as the percent % character. I rarely run into
that one though. The back-quote ` comes up a lot more often though, as
people have a habit of typing ` instead of ' when using contractions in
filenames. setdos/x-7 fixes that, but turns off the double-quoting that I
would like to function normally.

[r***@highfiber.com]

Double quotes around the variable are the way to go:

C:\>ver /r

4DOS 8.00 MS-DOS 5.50
4DOS Build 200 (Feb 27 2009)
DOS Revision A; DOS is in HMA

C:\>type testy.btm
@echo off

do f in ( c:\windows\*.bmp )
set fd=%@filedate["%f"]
set ft=%@filetime["%f",,s]

echo %fd %ft %f
enddo

C:\>testy.btm
03/31/03 06:00:00 c:\windows\Blue Lace 16.bmp
03/31/03 06:00:00 c:\windows\Coffee Bean.bmp
03/31/03 06:00:00 c:\windows\FeatherTexture.bmp
03/31/03 06:00:00 c:\windows\Gone Fishing.bmp
03/31/03 06:00:00 c:\windows\Greenstone.bmp
03/31/03 06:00:00 c:\windows\Prairie Wind.bmp
03/31/03 06:00:00 c:\windows\Rhododendron.bmp
03/31/03 06:00:00 c:\windows\River Sumida.bmp
03/31/03 06:00:00 c:\windows\Santa Fe Stucco.bmp
03/31/03 06:00:00 c:\windows\Soap Bubbles.bmp
03/31/03 06:00:00 c:\windows\Zapotec.bmp

C:\>

Post by FoxWolfie Galen
Try a name like "m 100 ani frame 02.gif"
  D:\PICS\GI.BAT [31]  File not found "C:\AnimWork\m"
  set fd=
  D:\PICS\GI.BAT [31]  File not found "C:\AnimWork\100"
  set fd=
  D:\PICS\GI.BAT [31]  File not found "C:\AnimWork\ani"
  set fd=
  D:\PICS\GI.BAT [31]  File not found "C:\AnimWork\frame"
  set fd=
  D:\PICS\GI.BAT [31]  File not found "C:\AnimWork\02.gif"
  set fd=
  D:\PICS\GI.BAT [32]  File not found "C:\AnimWork\m"
  set ft=
  D:\PICS\GI.BAT [32]  File not found "C:\AnimWork\100"
  set ft=
  D:\PICS\GI.BAT [32]  File not found "C:\AnimWork\ani"
  set ft=
  D:\PICS\GI.BAT [32]  File not found "C:\AnimWork\frame"
  set ft=
  D:\PICS\GI.BAT [32]  File not found "C:\AnimWork\02.gif"
  set ft=

This looks to me like the filename is being split at the spaces before
you ever reach those SET FD= and SET FT= commands. I think you have,
not an issue with the functions, but an error in the logic of your
program somewhere. Klaus's suggestion of debugging with SETDOS /Y1 is
likely to be helpful here. Also, if you're using a FOR loop, I would
suggest rewriting it to use DO instead. (Also note that @FILEDATE
doesn't take an S option. It probably doesn't have anything to do
with your problem, but I'd remove it on general principles.)

--
Charles Dye

[FoxWolfie Galen]

Post by r***@highfiber.com
This looks to me like the filename is being split at the spaces before
you ever reach those SET FD= and SET FT= commands. I think you have,
not an issue with the functions, but an error in the logic of your
program somewhere. Klaus's suggestion of debugging with SETDOS /Y1 is
likely to be helpful here.

It turned out to be a leftover setdos /x-7 .

Post by r***@highfiber.com
Also, if you're using a FOR loop, I would
suggest rewriting it to use DO instead.

One of the joys of cutting and pasting. I copied that line from the
filetime line and made the desired changes, but overlooked that. It appears
that 4DOS ignores the extra option, but it has now been removed.

FOR has that length limit that calling a subroutine solved. I'll rewrite
using a DO loop, and likely move the contents of the subroutine to the DO
loop.

Thanks for the advice from everyone.

[PCPete]

Post by FoxWolfie Galen

Post by r***@highfiber.com
This looks to me like the filename is being split at the spaces before
you ever reach those SET FD= and SET FT= commands. I think you have,
not an issue with the functions, but an error in the logic of your
program somewhere. Klaus's suggestion of debugging with SETDOS /Y1 is
likely to be helpful here.

It turned out to be a leftover setdos /x-7 .

Post by r***@highfiber.com
Also, if you're using a FOR loop, I would
suggest rewriting it to use DO instead.

One of the joys of cutting and pasting. I copied that line from the
filetime line and made the desired changes, but overlooked that. It
appears that 4DOS ignores the extra option, but it has now been removed.
FOR has that length limit that calling a subroutine solved. I'll rewrite
using a DO loop, and likely move the contents of the subroutine to the DO
loop.
Thanks for the advice from everyone.

I know I'm coming in to this conversation late, but this is one of the most
problematic and difficult issues I've ever come across in 4DOS/4NT/TCMD, and
still bugs the bejeezus out of me every week.

When we don't have that much control over filenames that must be handled and
processed, having the command interpreter not able to deal with a single
string of arbitrary (but still _legal_) complexity without crippling or
disabling (effectively) other functions is really frustrating.

I feel for you, and let me say, I've had some particularly close-to-the-limit
filenames that no command interpreter could handle (did someone say
unicode?). I've ended up hand-editing 5,000+ filenames instead of persisting
with the limitations of the string handling in JPSoft software.

Don't get me wrong, I love and use all varieties of 4DOS/NT/TCMD on dozens of
systems, but this kind of problem is a killer.

And thanks also for explaining about the limitations of the for construct,
that explains a LOT of the hair-pulling I've done over the past 18 months! I
ended up using DO loops simply because nothing else worked as documented when
non-"WORDFILE.TXT" filenames are in the mix.
Thanks all.
-PCPete

[E. S. Fabian]

PCPete wrote:
| I know I'm coming in to this conversation late, but this is one of
| the most problematic and difficult issues I've ever come across in
| 4DOS/4NT/TCMD, and still bugs the bejeezus out of me every week.
|
| When we don't have that much control over filenames that must be
| handled and processed, having the command interpreter not able to
| deal with a single string of arbitrary (but still _legal_)
| complexity without crippling or disabling (effectively) other
| functions is really frustrating.
|
| I feel for you, and let me say, I've had some particularly
| close-to-the-limit filenames that no command interpreter could
| handle (did someone say unicode?). I've ended up hand-editing 5,000+
| filenames instead of persisting with the limitations of the string
| handling in JPSoft software.

The fundamental problem is that the only thing COMMAND.COM and CMD.EXE and
therefor 4DOS+ knows is commands. Other interpreters, e.g., all versions of
Basic interpreters known to me, distinguish between the command string and
data. Long time ago 4DOS has added many additional special characters, when
those characters were not legal in file names. With the advent of VFAT and
esp. NTFS virtually all characters became valid in file names. Short of a
total redesign of the command language with the feature of being able to
distinguish strings in the command line as data vs. command, and a totally
new parser, the problem escalated. TCC10 introduced binary buffers, which at
least allow manipulating the data containing the syntactically significant
characters. By using the binary buffers to escape each character that needs
it, one can create strings that will pass through a single level of
interpretation without being misunderstood. Multiple levels of passing the
data are still troublesome. Of course, all of the *nix command interpreters
(shells) suffer a similar problem - a script (i.e., batch program) which
works if executed interactively may fail in executed from another script.

| And thanks also for explaining about the limitations of the for
| construct, that explains a LOT of the hair-pulling I've done over
| the past 18 months! I ended up using DO loops simply because nothing
| else worked as documented when non-"WORDFILE.TXT" filenames are in
| the mix.

FOR inherited its strange characteristics from COMMAND.COM... I prefer DO
loops in batch programs because of the much better control, easier
debugging, virtually unlimited loop length, etc.

[FoxWolfie Galen]

Post by PCPete
When we don't have that much control over filenames that must be
handled and processed, having the command interpreter not able to
deal with a single string of arbitrary (but still _legal_) complexity
without crippling or disabling (effectively) other functions is really
frustrating.

The worst characters for me have been the back-quote (`), the semi-colon
(;), the carat (^) the percent (%), and the space. Both the space and the
percent can be solved by using setdos /x, or enclosing the filenames in
double-quotes. The carat problem was harder to solve, as it was the command
separator by default. To solve that one. I changed my command separator to
something that cannot exist in a filename, and never had a problem with it
again. The escape character was always crtl-x (ASCII 24), so I figured I
would change my command separator to ctrl-y (ASCII 25). If I need to
separate commands, I simply use %+. I made that change six or seven years
ago, with no bad side effects.

This leaves just two legal ASCII filename characters that still give me
trouble - the back-quote and the semi-colon. Those come up in filenames
often enough to be a real nuisance. It would be great if we could simply
turn off the back-quote, without turning off the double-quote, using setdos
/x-7. Turning off the semi-colon would be nice too. Unfortunately, there
are no more numbers available for the setdos /x command. Maybe an
additional non-numeric character after the /x could bet set to take care of
these last two problems.

I created test files containing the following ASCII characters.

! @ # $ % & ^ ~ - _ ' ` ( ) { } . = + , ; [ ] space A-Z 0-9

These can all be legally created, open, saved, etc, in CMD, Explorer, or
any other Windows program. 4DOS can process all of these in an automated
fashion, except for

` and ;

By automated, I mean a batch file that can process filename containing
these characters, without the user needing to rename them first. That
simply isn't practical for a batch that is supposed to work with real files
from random sources.

Neither DOS nor Windows allow the creation of files containing any of these
characters, so they should never be able to occur in the wild.

< > | / \ ? * : "

Putting double-quotes (") around filenames containing either the semi-colon
(;) or the back-quote (`) does not allow them to be parsed. Working with
the SFN of a file containing a semi-colon can get around that character,
but working with the SFN of a file is not always desired or without
problems of its own. Nothing short of setdos /x-7 allows a back-quote to be
used in a filename, in my experience. Setting that, breaks the double
quotes though, which effectively breaks files containing spaces and other
characters that are even more common. I use 4DOS daily, and love it, but
this is one area where it isn't quite perfect.